testing on Raspberry Pi

[carterw]

I successfully installed moonshine on a Pi 5 and did some experimenting. I recorded a 5-second audio file of my voice and also there is a test file as part of the install.

The included test file gets transcribed in about 5 seconds from a cold start. Both that file and my voice file each take about 3 seconds after that first transcription. Not bad! I would like to try the streaming demo to see if it will keep up but I don't have a suitable microphone at present.

[evmaki]

Thanks for trying it out and reporting your results! The "cold start" delay when running useful-moonshine-onnx for the first time is likely just the time it takes for the models to be downloaded from HuggingFace hub. Once they are downloaded after your very first run, subsequent transcriptions should run more quickly.

[carterw]

Interesting, I didn't know that the models have to be downloaded at runtime. I assume this means that the transcription requires an internet connection and cannot run standalone?

[evmaki]

The MoonshineOnnxModel can be created by specifying either the model_name or models_dir:

• If you specify the model_name, then the model is automatically downloaded from the HuggingFace hub only once the first time you run it. Each subsequent run does not require an internet connection because the models have been downloaded already. This is what moonshine_onnx.transcribe uses under the hood when you pass it a string for the model name, like so:

from moonshine_onnx import transcribe

moonshine_onnx.transcribe("path/to/audio.wav", "moonshine/base")

• If you use models_dir instead, you can manually download the models from here and deploy them to the Raspberry Pi yourself. You create the MoonshineOnnxModel object yourself and pass it into moonshine_onnx.transcribe instead of a string, like so:

from moonshine_onnx import MoonshineOnnxModel, transcribe

model = MoonshineOnnxModel(models_dir="path/to/manually/downloaded/models/")
moonshine_onnx.transcribe("path/to/audio.wav", model)

[carterw]

I tried that and got this failure. Looks like it doesn't like the contents of preprocess.onnx?

$ python test2.py Traceback (most recent call last): File "/home/bill/env_moonshine/test2.py", line 8, in <module> model = MoonshineOnnxModel(models_dir=modelpath) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/bill/env_moonshine/lib/python3.11/site-packages/moonshine_onnx/model.py", line 28, in __init__ self.preprocess = onnxruntime.InferenceSession(preprocess) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/bill/env_moonshine/lib/python3.11/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 465, in __init__ self._create_inference_session(providers, provider_options, disabled_optimizers) File "/home/bill/env_moonshine/lib/python3.11/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 526, in _create_inference_session sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ onnxruntime.capi.onnxruntime_pybind11_state.InvalidProtobuf: [ONNXRuntimeError] : 7 : INVALID_PROTOBUF : Load model from /home/bill/moonshine/onnx/tiny/preprocess.onnx failed:Protobuf parsing failed.

The files in the 'tiny' directory are suspiciously small;
$ ll /home/bill/moonshine/onnx/tiny
total 16
-rw-r--r-- 1 bill bill 134 Nov 24 19:32 cached_decode.onnx
-rw-r--r-- 1 bill bill 133 Nov 24 19:32 encode.onnx
-rw-r--r-- 1 bill bill 132 Nov 24 19:32 preprocess.onnx
-rw-r--r-- 1 bill bill 134 Nov 24 19:32 uncached_decode.onnx

[carterw]

It looks like there was a problem with cloning from https://huggingface.co/UsefulSensors/moonshine/tree/main/onnx. I downloaded the files manually and it works.

[carterw]

I installed the requisites for the live_captions.py demo on my Pi 5 and I'm using a USB gaming headset. It worked right away. It keeps up with the speech input with no problem.

I'm amazed at the accuracy. I can read in long sections of text and often there are no errors. I can see how this could be useful in a lot of projects where you would be able to speak voice commands instead of using a keyboard or some other kind of controller.