Lightweight C++ template library that builds on top of TensorRT C++ API extending it with quality-of-life and safety improvements. Compile-time model architecture definition allows for catching dimension mismatches before execution!
This library is in early development stages (currently it's just PoC). Features may be missing and issues may occur!
- Compile time Module definition API
- Compile time data shape/type checks
- Better developer experience
- Predefined layers
- Flexible logger
- TensorRT container 23.05
- TensorRT 8.6.1
Tested on NVIDIA GTX 1060 6GB Pascal Architecture.
- Convolution Layer
- Split/Join Layers
- Weights loader
- Examples & Benchmarks
Build Image & Compile
docker buildx build -t *TAG* --platform *PLATFORM* --load .
Compile
RUN mkdir build
RUN cd build
RUN cmake ..
RUN make
Test
ctest
Generate Docs
After compilation in build directory:
make docs