You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit c030572
Browse filesBrowse the repository at this point in the historyBrowse files
Python runtime: enforce CUDA stream selection rules on the stream itself (#24327)
## What is wrong today
`executorch.runtime.cuda` (added in #24273) lets Python code choose the
CUDA stream that the CUDA and TensorRT delegates run on. Some rules were
checked on what a wrapper says, not on the stream, and some docs
promised too much:
- A stream on GPU 1, given as `cuda.ExternalStream(ptr)` or
`torch.cuda.ExternalStream(ptr)`, was accepted, then failed in the
delegate.
- `cudaStreamLegacy` (handle 1) got past the refusal of the legacy
default stream (handle 0).
- The docs said a closed stream cannot be selected, a dropped scope
object ends its block, and a call inside a block waits only for its
stream. None of these always holds.
## What this change does
- In CUDA 13 builds, the native scope asks CUDA for the stream's device
when a block starts (`cudaStreamGetDevice`) and refuses any device but
0. Every CUDA wheel built from main is CUDA 13. CUDA 12 builds keep only
the wrapper check, since they can load a runtime older than 12.8, which
lacks that function. On a stream that is capturing a CUDA graph, the
query is skipped: CUDA refuses it there, and the refusal ends the
caller's capture.
- The native scope refuses handles 0 and 1, `None`, an exit from another
thread, and a second entry. After refusing an exit out of order, it can
be entered again once its block has ended.
- `_CallerStreamScope.can_check_device` is a build-time attribute, not a
CUDA call.
- The docs say what is enforced, which waits a block shortens, and which
steps still wait longer (outputs whose layout changes, loading CUDA
weights).
- New tests: a block inside `torch.cuda.graph()` leaves the graph
intact, a refused native scope can be entered again, and CUDA 13 CI
checks that the device query is built in.
- `unittest-cuda` no longer starts for a change to `runtime/cuda.py`
alone; nothing it runs imports it.
## Tests
On Linux with A100 GPUs, bindings built with CUDA 13.2:
- `test_pybindings_cuda_dlpack.py` passes in both output modes, with a
second GPU so the device test runs.
- Each new check fails without its fix: capture check removed, refused
exit keeping its mark, device query compiled out, and the bindings from
before these changes.
- A library built against CUDA 12.8 that calls `cudaStreamGetDevice`
fails to load next to the CUDA 12.6 runtime.
Not run on Windows, ROCm, or a real CUDA 12 build.
0 commit comments