New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support for ONNX models & INT8 quantization #94

Open

bkaruman opened this issue Jun 11, 2021 · 2 comments

bkaruman commented Jun 11, 2021

Thanks for the excellent toolkit.
Are there plans to support ONNX models? It would be nice to see some speed-up data on quantized(INT8) attention.

Contributor

feihugis commented Jun 12, 2021

@bkaruman We did not have a plan to support ONNX models yet.

Contributor

JiushengChen commented Jun 16, 2021

There is tool to convert PyTorch model to ONNX model, it should be applicable to FastSeq as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment