Skip to content

Conversation

@fsossai
Copy link

@fsossai fsossai commented Sep 7, 2021

Benchmarks of models with ONNX Runtime and SOFIE

SOFIE is designed for fast inference, that's why having a reference benchmark with the state-of-the-art ONNX processing framework (ONNX Runtime) comes in handy for asserting the performance of the models emitted by SOFIE.

Integrating ONNX Runtime

The official repository still doesn't provide a CMake script for automatically finding ORT in the system, that's why I wrote a basic package finder.

  • CMake package finder (a.k.a. FindONNXRuntime.cmake).
  • GBenchmark for ONNX models with ONNXRuntime libraries.
  • Manual GBenchmark for models emitted by SOFIE, Linear models only.
  • Automating the compilation of models with SOFIE.

Notes

  • For every ONNX model in input_models a gbenchmark is automatically created.

  • For every .hxx file in input_models/compiled a gbenchmark for SOFIE is automatically created.
    All of this happens by means of BENCHMARK_CAPTUREs configured a CMake script.

@fsossai fsossai marked this pull request as ready for review October 21, 2021 15:36
@lmoneta
Copy link
Member

lmoneta commented Nov 25, 2021

This PR is replaced by #239. The initial commits are moved to the new PR.

@lmoneta lmoneta closed this Nov 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants