Skip to content

Conversation

admiralakber
Copy link
Contributor

Hey mate,

Simple yet high value add, I thought I should upstream it. I have tested this on Linux and Android. The OpenCL backend is much more reliable than Vulkan.

✨ Summary

  • Introduce an opencl feature for both crates, package ggml-opencl sources, and enable GGML_OPENCL in build.rs with per-OS linking to the OpenCL loader. Keeps changes consistent with existing CUDA/Vulkan patterns.

🔎 Diagnosis

🚀 Changes

  • New feature flag on llama-cpp-2/Cargo.toml and added new globbing to llama-cpp-sys-2/Cargo.toml
  • Little tweak to the llama-cpp-sys-2 README.md should help SEO if that matters anymore
  • Most of the work in llama-cpp-2-sys/build.rs - matched the existing patterns

☁️ What you’ll need to do

  • Just build with the opencl feature flag

🧪 Smoke test

cargo run -p llama-cpp-2 --example usage --features opencl -- ${PATH_TO_MODEL} --verbose 

✅ Acceptance

  • If someone else can do a manual test that would be great. A Linux box with a dediated GPU would be strong validation.
    • Note: It might be that the llama.cpp OpenCL backend is only supported for Android which I can report is working great.
  • A Windows or Mac build just to confirm the rust link commands in build.rs are correct.

Introduce an `opencl` feature for both crates, package ggml-opencl sources,
and enable GGML_OPENCL in build.rs with per-OS linking to the OpenCL loader.
Keeps changes consistent with existing CUDA/Vulkan patterns.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants