Deepseek models not working with LLMFarm

[miladyalda]

Hello,

I have trying to use "DeepSeek-Coder-V2-Lite-Instruct-Q6_K.gguf" with LLAMA Farm, I tried with the app from TestFlight and source code from Github. it fails for both.

Any suggestion how to make this works?

[guinmoon]

Hi. Please tell me what hardware you run on and send me a link to the model.

[miladyalda]

Hello Guinmoon,

https://huggingface.co/bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUF/tree/main

I used the Q6_K.

The hardware:
Model Name: MacBook Pro
Model Identifier: MacBookPro18,3
Model Number: MKGP3KS/A
Chip: Apple M1 Pro
Total Number of Cores: 8 (6 performance and 2 efficiency)
Memory: 16 GB