Error: cannot find tensor lm_head.weight

[malikwirin]

I am having an issue with deploying Qwen2.5-Coder with the WIP program Cake.

The following error happens when using the models Qwen2.5-Coder-3B and Qwen2.5-Coder-3B-Instruct from your hugginface repos:

[2024-11-18T15:36:29Z INFO ] [Master] dtype=F16 device=Cpu mem=6.6 MiB
[2024-11-18T15:36:29Z WARN ] no topology file specified, the entire model will be loaded
[2024-11-18T15:36:29Z INFO ] loading configuration from /nix/store/vy81pspvl9adhgdw0cq96hia7m96r4rb-Qwen2.5-Coder-3B-Instruct/config.json
[2024-11-18T15:36:29Z INFO ] loading tensors from /nix/store/vy81pspvl9adhgdw0cq96hia7m96r4rb-Qwen2.5-Coder-3B-Instruct/model.safetensors.index.json ...
[2024-11-18T15:36:29Z INFO ] loading embeddings ...
[2024-11-18T15:36:30Z INFO ] loading lm_head ...
Error: cannot find tensor lm_head.weight

What is meant with tensor lm_head.weight and which codebase could be at fault?

[cyente]

i am not sure if Cake support qwen series models. we recommend you deploy qwen models with vllm.