DeepDeploy-Toolkit

DeepDeploy-Toolkit is designed for deploying deep learning models across diverse hardware and frameworks.

This repository provides deployment examples using Python and C++. Typically, models are tested and evaluated with Python on PCs before deployment using C++ on devices.

Features

• Supported Hardware:

  • x86 CPU
  • Nvidia GPU
  • ARM CPU
  • ARM GPU
  • ARM RKNN NPU

• Supported Frameworks:

  • OnnxRuntime
  • TensorRT
  • MNN
  • RKNN-Toolkit

Note: Development is ongoing.

Requirements

Python

• Python 3.7
• OpenCV 4.5
• PyTorch 1.10
• OnnxRuntime/OnnxRuntime-GPU 1.9.0
• RKNN-Toolkit 1.7.1
• MNN 1.1.6

C++

• CMake 3.19.1
• OpenCV 4.5.0
• TensorRT 8.0.3.4
• MNN 1.2.0
• OnnxRuntime 1.9.0

Acknowledgements

The following projects have been instrumental in building this toolkit:

• PyTorch
• TensorRT
• OnnxRuntime
• MNN
• RKNN-Toolkit
• RKNN NPU