Is it possible to provide a LLVM independent build?

[Neutron3529]

I tried to publish another cuda crate without llvm or specific rust compiler support:

https://github.com/Neutron3529/cuda_min

Such crate works very well, and thus I have some doubt: why a specific llvm version and rust version is needed?

It seems that

1. llvm-bitcode-linker could compile rust code into PTX
2. nvidia driver could load PTX code directly. (no need to install cudart, nvidia's driver provides interfaces such as cuInit.)

I just wonder why a specific llvm version even a specific rustc version is required. We should have the ability to compile a cuda program easier.

[LegNeato]

@brandonros did some experiments with emitting llvm ir and then using llvm-bitcode-linker I believe. Care to comment on if this path is viable?

[brandonros]

@LegNeato I did not feel there was a lot of good warm than reception on that approach.

This repo is really cool because you are taking over somebody's extremely hard to do rustc_codegen approach from LLVMv7

Is there any appetite to consider alternatives?

I get that Blackwell is very early and a very small piece of the market, but if we are forward looking, I would guess that within the next two years, the majority of the market will be using this new LLVM v19 version of NVVM/etc

I would like to sync with you on this, I am sure you were busy, but I basically came up with two different approaches

One of them is the one that I PR'd where I fumbled through making a new rustc_codegen

Another is where I give up a ton of flexibility related to address space but I was able to do LLVM IR -> NVVM which I would really like to show you and get some feedback on whenever you have time

[LegNeato]

I wonder if we can statically link it in, so at least users don't need it installed.

[LegNeato]

@brandonros I took a look at your approach, my main concern is I don't understand the tradeoffs. I also expect that would be harder to get into the compiler proper. But again, I don't really know. I would imagine if we are in rustc we can distribute nvvm or compiled llvm7 in its own component and we'd have to probably do the same for LLVM v19 as I don't think nvidia is going to track LLVM versions with NVVM like rustc does (but who knows!). So this llvm stuff is only a problem because we don't have all the rustup / rustc infra as an external codegen backend.

[LegNeato]

What I dont want to do is reimplement most things that NVIDIA handles. That's a fools errand...we want to swim with them rather than try to reimplement everything (minus some bits that maybe are a pain or need to be rust-native).

[devillove084]

I'm trying to use the precompiled libraries available here https://github.com/awakecoding/llvm-prebuilt/releases to rematch parts of the Rust wrapper and the relevant code generation. I understand your concerns, but this task also requires a good understanding of some APIs in LLVM and NVVM. Perhaps I should write a document to summarize the current work and then simultaneously address the issues of API matching and code adaptation. What do you think? @LegNeato

[LegNeato]

Sure, whatever works for you. I am not familiar with this part of the project so don't have any strong opinions.

I think ideally we:

1. Support both llvms and switch depending on target features (similar to "native" on CPU)
2. Compile (both?) llvm into the codegen backend statically so end users don't need to deal with anything
3. Have a docker container(like we do now) for rust-cuda developers to develop in)

[devillove084]

@LegNeato

1. I think supporting multiple LLVM versions simultaneously is feasible, but we can start with the mainstream ones first, such as versions 10, 14, 16/18.
2. Regarding the second point, do you mean that both the NVVM backend and the LLVM backend should be statically compiled into the codegen backend?
3. No problem, I'm already working on this.

[LegNeato]

@devillove084

1. we should only need 7.1 (for older nvvm) and 18 (for newer nvvm) from the CUDA side (and rustc will have a third for CPU side, and that is a moving target). I'm not sure how often nvidia will update nvvm going forward.

2. Yes, we want end users to not worry about nvvvm/llvm, similar to how in normal rust you just get llvm with rustc.

3. The current docker stuff is pushed here: https://github.com/orgs/Rust-GPU/packages?repo_name=Rust-CUDA. The Dockerfiles are in the repo.

[devillove084]

@LegNeato Okay, I get it. I'll take it step by step, making sure that we know what we're doing at each stage—something that's testable, explainable, and backed by solid reasoning.