Review notes from a build-discovery pass:

Thanks for tackling this. The motivation makes sense: CUDA discovery is currently duplicated across build scripts, and the classic /usr/local/cuda/include + /usr/local/cuda/lib64 assumption breaks real conda/micromamba and NVIDIA HPC SDK installs.

The new openinfer-build crate is a good direction. Centralizing find_package, CUDA header candidates, and CUDA library candidates makes the behavior easier to test and reason about. The unit tests for classic, conda-style, and HPC SDK layouts are also useful.

I do not think this fully closes #342 yet, though. The issue lists several CUDA discovery sites, but this PR only migrates part of them. A few active build paths still keep their own CUDA assumptions:

• openinfer-comm/crates/openinfer-comm-a2a-kernels/build.rs:31 still hardcodes /usr/local/cuda/lib64.
• openinfer-comm/crates/openinfer-comm-torch-lib/build.rs:29 still hardcodes /usr/local/cuda/include.
• openinfer-cupti/build.rs:3 still has separate CUDA root/include/lib logic.
• kvbm/kvbm-kernels/build.rs:369 still has its own CUDA library path handling.

There is also one partial fix inside openinfer-kernels: the link search path now uses openinfer_build::link_cuda, but the compile/include side still assumes $CUDA_HOME/include at openinfer-kernels/build.rs:1139 and openinfer-kernels/build.rs:1102. That means a conda layout with headers under targets/<arch>-linux/include can still fail for generated Triton wrapper compilation.

My suggestion is to make openinfer-build expose one higher-level CUDA discovery result, for example root, include dirs, lib dirs, stub lib dirs, and nvcc path, then migrate all CUDA build scripts to consume that instead of each caller composing pieces manually.

I verified:

• cargo test --release -p openinfer-build --lib
• cargo fmt --all --check
• cargo metadata --no-deps --format-version 1

I could not run the CUDA build locally because nvcc is not on PATH on this host.

Thanks for the careful review(and verification)! Agreed on these points, and the include-side catch is a real gap. I'll push a patch to this PR later :)

Pushed the follow-up covering these points.

Discovery now lives in one openinfer_build::CudaToolkit: root, nvcc, existing include dirs, and existing lib dirs in link order, plus header_dir(h) for host-compiler -I flags and link_search() / link_search_stubs(). All seven sites consume it: openinfer-kernels, cuda-sys, cudart-sys, a2a-kernels, torch-lib, cupti, and kvbm-kernels.

One behavior change to flag: kvbm-kernels used to check CUDA_PATH before CUDA_HOME, the opposite of the rest of the workspace; CudaToolkit resolves CUDA_HOME first, which only matters when both are set to different roots. Happy to special-case it if the old precedence was intentional. cupti also hardcoded targets/x86_64-linux, silently broken on aarch64; this fixes that in passing.

Before (conda toolkit, the include-side gap you predicted, reproduced with the CPATH workaround removed):

=B1= kernels triton wrapper host-cc compile
triton_flash_attention_prefill_hd256.cd26a890_.c:6:10: fatal error: cuda.h: No such file or directory
=B2= torch-lib hw-cuda
.../torch/include/ATen/cuda/CUDAContextLight.h:9:10: fatal error: cuda_runtime_api.h: No such file or directory
=B3= comm bins real link (blocked behind the kernels failure)
error: failed to run custom build command for `openinfer-kernels v0.1.0 (...)`

After — with the CPATH/LIBRARY_PATH workarounds deleted, kernels, torch-lib (hw-cuda), the openinfer-comm binaries (first full link of pplx_a2a_bench, with gdrapi/ibverbs/cudart all resolved), cupti, and kvbm-kernels all build on the conda machine and the golden gate runs green; layout unit tests pass on all three hosts:

== Single GPU (x86_64, sm_89), conda toolkit, CPATH workaround deleted ==
=A1= openinfer-build unit tests
test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
=A2= kernels (triton wrapper via header_dir)
    Finished `release` profile [optimized] target(s) in 35.88s
=A3= torch-lib hw-cuda
    Finished `release` profile [optimized] target(s) in 9.40s
=A4= comm bins real link
    Finished `release` profile [optimized] target(s) in 56.35s
=A5= cupti
    Finished `release` profile [optimized] target(s) in 0.11s
=A6= kvbm-kernels
    Finished `release` profile [optimized] target(s) in 9.14s
=A7= golden gate full run
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 37.92s

== Dual-GH200 (aarch64, sm_90), HPC SDK toolkit, LIBRARY_PATH unset ==
=G1= openinfer-build unit tests
test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.04s
=G2= kernels relink (gate --no-run)
    Finished `release` profile [optimized] target(s) in 2m 15s

== CUDA-less dev host ==
test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s