Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

有相关onnx模型转换与调用的支持嘛? #172

Open
Gmgge opened this issue Jul 18, 2022 · 0 comments
Open

有相关onnx模型转换与调用的支持嘛? #172

Gmgge opened this issue Jul 18, 2022 · 0 comments

Comments

@Gmgge
Copy link

Gmgge commented Jul 18, 2022

你好,非常感谢你们开源如此棒的工作

我在尝试将模型转换为onnx,以使用推理框架onnxruntime。
我进行了以下的工作
1.参考huggingface/transformers,调用模型转换将预训练模型转换为onnx,参考教程在此
运行脚本为:
python -m transformers.onnx --model "/project/gm/project/transformers/albert-chinese-tiny-train/" albert-chinese-tiny-train
结果为:
Some weights of the model checkpoint at /project/gm/project/transformers/albert-chinese-tiny-train/ were not used when initializing AlbertModel: ['predictions.dense.bias', 'predics.decoder.bias', 'predictions.LayerNorm.weight', 'predictions.LayerNorm.bias', 'predictions.decoder.weight', 'predictions.dense.weight']

  • This IS expected if you are initializing AlbertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClasBertForPreTraining model).
  • This IS NOT expected if you are initializing AlbertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification modeClassification model).
    Some weights of AlbertModel were not initialized from the model checkpoint at /project/gm/project/transformers/albert-chinese-tiny-train/ and are newly initialized: ['albert.pooler.bias']
    You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
    Using framework PyTorch: 1.8.1+cu102
    Validating ONNX model...
    -[✓] ONNX model output names match reference model ({'last_hidden_state'})
    - Validating ONNX Model output "last_hidden_state":
    -[✓] (2, 8, 312) matches (2, 8, 312)
    -[✓] all values close (atol: 1e-05)
    All good, model saved at: albert-chinese-tiny-train/model.onnx

2.参考transformers中对onnx的模型调用,结果提示参数错误
session = InferenceSession("onnx/model.onnx")
inputs = tokenizer("Using DistilBERT with ONNX Runtime!", return_tensors="np")
outputs = session.run(output_names=["last_hidden_state"], input_feed=dict(inputs))

有相关提示或者文档嘛?我阅读了本项目的文档,但是没有找到相关信息,不知道是否因为有些遗漏。

如果有相关的提示或者样例代码以参考,非常感谢

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant