Can no longer force LLAMACPP_BACKEND to rocm following upgrade to 10.2

[derzahla]

Platform

Linux/Ubuntu

Lemonade Version

10.2

GPU / APU Model

Radeon 760M

Component

llama.cpp

Bug Description

Previously, on v10.0.1, I was able to download your latest build of llama.cpp-rocm, copy to /var/lib/lemonade/.cache/lemonade/bin/llamacpp/rocm and force lemonade server to use it by setting the following options in the config:

LEMONADE_LLAMACPP_BACKEND="rocm"
LEMONADE_LLAMACPP_ROCM_BIN="/var/lib/lemonade/.cache/lemonade/bin/llamacpp/rocm/llama-server"

It worked well, gave a slight performance boost and allowed me to run models that vulkan couldnt handle for some reason.

Now with v10.2, this failed immediately when trying to load a model with:

Failed to load model: Failed to load model 'user.Qwen3-30B-A3B-Instruct-2507-REAM-heretic-GGUF': 'llamacpp.backend' must be one of: auto, vulkan, cpu

How can I achieve the same functionality I had before?

Steps to Reproduce

1. start lemonade server
2. attempt to load a model

Expected vs Actual Behavior

No response

Log Output

Additional Context

No response

[jeremyfowers]

Please see https://github.com/lemonade-sdk/lemonade/wiki/Migration#v10x---v101

lemonade config set llamacpp.backend=rocm

[derzahla]

@jeremyfowers Thanks for the reply. I get the same error with this method:

# lemonade config set llamacpp.backend=rocm
Error setting config: 'llamacpp.backend' must be one of: auto, vulkan, cpu

# lemonade recipes
Recipe              Backend     Status          Message/Version                               Action
----------------------------------------------------------------------------------------------------------------------------------------------------
flm                 npu         unsupported     Requires AMD XDNA 2 AMD NPU                    -
kokoro              cpu         installed       b16                                            -
llamacpp            cpu         installed       b8668                                          -
                    metal       unsupported     Requires macOS                                 -
                    rocm        unsupported     Unsupported GPU: gfx110X                       -
                    system      unsupported     llama-server not found in PATH                 -
                    vulkan      update_required Backend update is required before use.         lemonade backends install llamacpp:vulkan
ryzenai-llm         npu         unsupported     Requires Windows                               -
sd-cpp              cpu         installed       master-506-1f30df9                             -
                    rocm        unsupported     Unsupported GPU: gfx110X                       -
whispercpp          cpu         installed       v1.8.2                                         -
                    npu         unsupported     Requires Windows                               -
                    vulkan      installed       v1.8.2                                         -
----------------------------------------------------------------------------------------------------------------------------------------------------

I think the problem is, because it detects I have a gfx110X GPU, it forces rocm to be an unsupported backend, even though I am overriding the llama-server binary with the lemonade-sdk build of llama.cpp-rocm, which DOES support gfx110X. any way around this?

[jeremyfowers]

@superm1 looks like ROCm GPU support identification is failing on Ubuntu for this user, can you help?

[derzahla]

@jeremyfowers @superm1 just to be clear, I have a integrated gfx1103. I was able to get everything working again by hacking some on src/cpp/server/system_info.cpp and src/cpp/server/server.cpp and building. But it would be great if it didnt try to completely shutdown rocm for the gfx1103, especially if im providing an override path for the binary

[superm1]

Let me understand - when you build your own llama.cpp binary, what ROCm stack are you compiling it against?

[derzahla]

@superm1 sorry if i wasnt clear. I am not building my own llama.cpp binary, i am simply downloading and using lemonade-sdk's build here https://github.com/lemonade-sdk/llamacpp-rocm

[superm1]

Can you share the specific diff you're using to make things work? gfx1103 is supposed to map out to gfx110x.

Also what hardware do you actually have?
https://github.com/torvalds/linux/blob/master/drivers/gpu/drm/amd/amdkfd/kfd_device.c#L419

[superm1]

My current theory is that your GPU is recognizing wrong in Lemoande because gfx110x is for dGPUs but you have an APU.

[superm1]

See if this helps: #1648

[derzahla]

Yes, i have an iGPU. Ryzen 760m. I will give you PR a try!(while i got llama.cpp to work sd-cpp still isnt running through lemonade). Thanks!

[derzahla]

So this seems to work great with llama.cpp. sd-cpp seems like its going to work but errors out after loading the models for some reason. I can run the sd-cli binary fine and that works so im not sure what the issue could be but ill keep looking. very welcome this PR though.

[superm1]

That's great. After I land this and ROCm channels (#1460) let's revisit your SD issue if it's still present.

[derzahla]

Awesome. thank you Sir!