Hailo Learning

Exploring capabilities of Hailo 10H ( on Raspberry Pi 5)

Setup

My setup:

cd 
mkdir projects && cd projects

Then build rest from source:

• Hailo PCIe driver: git clone https://github.com/hailo-ai/hailort-drivers
• HailoRT:

Examples need rust for tokenizer:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
. "$HOME/.cargo/env"

Install: git clone https://github.com/hailo-ai/hailort.git cd hailort cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release -DHAILO_BUILD_EXAMPLES=1 -DHAILO_BUILD_CLIENT_TOKENIZER=ON sudo cmake --build build --config release --target install hailort_examples sudo ldconfig

Note: you need to install host tokenization with -DHAILO_BUILD_CLIENT_TOKENIZER=ON, otherwise you'll get: [HailoRT] [error] Host-tokenization is not enabled in lib compilation

Commands

hailortcli logs runtime --device-id 0001:03:00.0
hailortcli scan

Examples:

Get .hef!

https://hailo.ai/products/hailo-software/model-explorer/generative-ai/

Qwen2 1.5B:

curl https://dev-public.hailo.ai/v5.1.1/blob/Qwen2-1.5B-Instruct.hef --output Qwen2-1.5B-Instruct.hef 

Compiled examples are at:

~/projects/hailort/build/hailort/libhailort/examples/genai

Two Hailos:

cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release 
sudo cmake --build build --config release

Device states?

D0? D3