Skip to content
Open
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
3d3327d
address doc CI issues
hughperkins Jun 24, 2026
dde771b
[CI] Fold reading-order (no-forward-reference) check into doc-quality
hughperkins Jun 24, 2026
aafcc41
[Doc] Document reading-order rule in doc-quality check description
hughperkins Jun 24, 2026
899600e
[CI] Simplify doc-quality RULE 3: drop reworked cap and double-flag note
hughperkins Jun 24, 2026
5f7aeb0
Merge remote-tracking branch 'origin/hp/doc-quality-with-ordering' in…
hughperkins Jun 25, 2026
57f07bd
fix doc ci failures
hughperkins Jun 26, 2026
715d454
Merge branch 'main' into hp/fixup-graph-doc
hughperkins Jun 26, 2026
cc8ea8a
doc tewaks for ci
hughperkins Jun 26, 2026
2fc9569
Merge remote-tracking branch 'origin/main' into hp/fixup-graph-doc
hughperkins Jun 26, 2026
ee7754c
Apply suggestion from @graphite-app[bot]
hughperkins Jun 26, 2026
1af37b9
Merge branch 'main' into hp/fixup-graph-doc
hughperkins Jun 26, 2026
04f2f0e
address doc CI
hughperkins Jun 26, 2026
a1d925b
address doc ci issues
hughperkins Jun 26, 2026
04f1f09
address some comments
hughperkins Jul 2, 2026
1e2e53b
Add Advanced section to graph doc explaining kernel launch latency
hughperkins Jul 2, 2026
e323513
Trim python-side latency bullet in graph doc Advanced section
hughperkins Jul 2, 2026
e78e40e
Trim c++-side latency bullet in graph doc Advanced section
hughperkins Jul 2, 2026
deb8969
Trim gpu/driver-side latency bullet in graph doc Advanced section
hughperkins Jul 2, 2026
b6f9732
Use 'launch' instead of undefined 'replay' in graph doc Advanced section
hughperkins Jul 2, 2026
1801688
Add latency hiding section to graph doc Advanced section
hughperkins Jul 2, 2026
b192ca8
Reword 'This is why' to 'So' in graph doc latency hiding section
hughperkins Jul 2, 2026
b58a08b
Clarify subject in graph doc latency hiding section ('It' -> 'Kernel …
hughperkins Jul 2, 2026
80eb935
Soften claim in graph doc latency hiding section ('directly increases…
hughperkins Jul 2, 2026
d634672
Soften unsupported-hardware claim in graph doc graph_do_while section
hughperkins Jul 2, 2026
c14a85e
Remove closing summary paragraph from graph doc graph_do_while section
hughperkins Jul 2, 2026
cfde9ce
Soften graph_do_while subsection heading in graph doc ('helps' -> 'ca…
hughperkins Jul 2, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 24 additions & 3 deletions .github/workflows/check_doc_quality.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ jobs:
/tmp/changed_docs.txt contains a newline-separated list of Markdown (.md) doc files (paths
relative to the repo root) that were added or modified in this PR. For EACH file in that
list, read the FULL current contents of the file (not just the diff) and review it against
the two rules below.
the three rules below.

Target audience: an *end user* of Quadrants. That is, someone who writes GPU programs in
Python using the Quadrants library (`@qd.kernel`, `@qd.func`, ndarrays, fields, etc.).
Expand Down Expand Up @@ -118,6 +118,25 @@ jobs:
VIOLATION: internal / non-end-user material (subject to the carve-outs above) appears
outside of such a clearly-marked advanced / under-the-hood section.

RULE 3 — Reading order (no forward references).
Within a single doc, information should be ordered so a first-time reader can follow it
top-to-bottom in one pass, without jumping ahead. A FORWARD REFERENCE is when a passage
depends on a concept, parameter, behavior, code element, or result that is only introduced
LATER in the SAME file, such that the reader cannot understand the current passage without
first reading the later one.
The following are NOT order violations:
- A brief overview / roadmap near the top that previews upcoming sections ("this guide
covers A, then B") — signposting the structure, not a dependency: the reader does not
need to understand A or B yet to keep reading.
- Optional "for more detail, see <section> below" pointers, where the current passage is
fully understandable WITHOUT following them. The test is: MUST read ahead to understand
(violation) vs more depth merely available later (fine).
- Backward references (to something earlier in the same file) are always fine.
- References to OTHER docs/files; this rule is about ordering WITHIN a single file only.
- Conventional preamble ordering (e.g. Prerequisites -> Installation -> Usage).
VIOLATION: a passage cannot be understood by a first-time reader at the point they reach
it, because what it relies on is only introduced later in the same file.

Read each listed file in full before judging it. You may open any other file in the repo
(e.g. linked docs) to verify whether a term is defined elsewhere. Do NOT modify any files.
Be precise and conservative: only flag clear violations, not borderline cases. Judge each
Expand All @@ -126,8 +145,10 @@ jobs:
If there are NO violations, your final output must start with the word PASS.
If there ARE violations, your final output must start with the word FAIL, followed by a list
of violations (up to 10), one per line, in the format:
<filepath>: [term|scope]: <brief description>
where [term] is a Rule 1 violation and [scope] is a Rule 2 violation.
<filepath>: [term|scope|order]: <brief description>
where [term] is a Rule 1 violation, [scope] is a Rule 2 violation, and [order] is a Rule 3
violation (for [order], cite both line numbers — what is used at line N is only introduced
at line M).
PROMPT
)" --model claude-4.6-opus-high-thinking --mode ask --output-format text --trust)

Expand Down
4 changes: 2 additions & 2 deletions docs/source/user_guide/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,9 +180,9 @@ The agent reports up to 5 violations, each annotated with the host file's hotnes

### Doc quality check (`check_doc_quality.yml`)

Uses an AI agent to review documentation changes for an end-user audience (someone writing Quadrants kernels in Python, not a compiler engineer). For each `docs/**/*.md` file added or modified in the PR, it reads the entire current file (not just the diff) and checks two things: (1) **undefined terms** — a term a typical user is unlikely to know (specialized or internal jargon, project-specific abbreviations) must be defined at its first use in that file, either inline or via a link to a doc that defines it; (2) **end-user relevance** — internal / implementation / contributor-only material must be confined to a clearly-marked section whose heading contains "Advanced", "Under the hood", "Internals", or "Implementation".
Uses an AI agent to review documentation changes for an end-user audience (someone writing Quadrants kernels in Python, not a compiler engineer). For each `docs/**/*.md` file added or modified in the PR, it reads the entire current file (not just the diff) and checks three things: (1) **undefined terms** — a term a typical user is unlikely to know (specialized or internal jargon, project-specific abbreviations) must be defined at its first use in that file, either inline or via a link to a doc that defines it; (2) **end-user relevance** — internal / implementation / contributor-only material must be confined to a clearly-marked section whose heading contains "Advanced", "Under the hood", "Internals", or "Implementation"; (3) **reading order** — information must be ordered so a first-time reader can follow the file top-to-bottom in one pass, without forward references (a passage that can only be understood by reading something introduced later in the same file).

The following do not count as violations: references to public APIs that the author links to their docs and/or labels as public; brief reader-directed pointers suggesting the reader could contribute upstream or file an issue; the core public API vocabulary a user already knows (e.g. `@qd.kernel`, `@qd.func`, `qd.Template`, fields, ndarrays). The agent reports up to 10 violations. This check is delayed by 30 minutes, to avoid running repeatedly if multiple commits pushed with a short delay between each.
The following do not count as violations: references to public APIs that the author links to their docs and/or labels as public; brief reader-directed pointers suggesting the reader could contribute upstream or file an issue; the core public API vocabulary a user already knows (e.g. `@qd.kernel`, `@qd.func`, `qd.Template`, fields, ndarrays); and, for the reading-order check, an overview/roadmap near the top, optional "see below" pointers (where the current passage still reads fine on its own), and backward references. The agent reports up to 10 violations. This check is delayed by 30 minutes, to avoid running repeatedly if multiple commits pushed with a short delay between each.

### PR change report (`pr_change_report.yml`)

Expand Down
48 changes: 23 additions & 25 deletions docs/source/user_guide/graph.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,6 @@

Graphs reduce kernel launch overhead by capturing a sequence of GPU operations into a graph, then replaying it in a single launch.

## Backend support

`graph=True` and `graph_do_while` run on every backend. They are *hardware accelerated* on CUDA (via CUDA graphs) and AMDGPU (via HIP graphs); `graph_do_while` additionally requires CUDA SM 9.0+ / Hopper for its hardware-accelerated path. On other backends, `graph=True` is silently ignored and the kernel runs via the normal launch path, and `graph_do_while` falls back to a host-side do-while loop that copies the condition value GPU → host each iteration (causing a pipeline stall). `qd.checkpoint` gating runs entirely on the device on every GPU backend; only the CPU backend uses host-side gating.

| Feature | `qd.cuda` SM 9.0+ | `qd.cuda` < SM 9.0 | `qd.amdgpu` | `qd.metal` | `qd.vulkan` | `qd.cpu` |
| --- | --- | --- | --- | --- | --- | --- |
| `graph=True` | hardware accelerated | hardware accelerated | hardware accelerated | runs (no acceleration) | runs (no acceleration) | runs (no acceleration) |
| `qd.graph_do_while` | hardware accelerated | host fallback | host fallback | host fallback | host fallback | host fallback |
| `qd.checkpoint` | GPU-side | GPU-side | GPU-side | GPU-side | GPU-side | host-side |

AMDGPU `graph_do_while` falls back to the host-side loop because HIP does not currently expose conditional / while graph nodes (as of ROCm 7.2).

Nested and sibling `graph_do_while` loops (and mixing `graph_do_while` with top-level `for`-loops) are **experimental** for now — see [Nested loops and mixing with for-loops](#nested-loops-and-mixing-with-for-loops).

## Basic usage

Add `graph=True` to a `@qd.kernel` decorator:
Expand Down Expand Up @@ -44,14 +30,14 @@ my_kernel(x, y) # first call: builds and caches the graph
my_kernel(x, y) # subsequent calls: replays the cached graph
```

This works the same way on CUDA and AMDGPU. The cache is keyed per (compiled-kernel-specialization, launch-id), so different template instantiations (different field bindings, etc.) get their own cached graph.
This works the same way on CUDA and AMDGPU.

### Restrictions

- **No struct return values.** Kernels that return values (e.g. `-> qd.i32`) cannot use graphs. An error is raised if `graph=True` is set on such a kernel.
- **Primal kernels only.** The `graph=True` flag is applied to the primal (forward) kernel only, not its adjoint. Autodiff kernels use the normal launch path.
- **Primal kernels only.** The `graph=True` flag is applied to the primal (forward) kernel only, not its adjoint (backward). [Autodiff](autodiff.md) kernels use the normal launch path.
- **Device-resident ndarrays.** Graph mode bakes device pointers into the cached graph, so all ndarray arguments must be on the GPU. Passing a host-resident ndarray raises an error.
- **`qd_stream` is incompatible** with `graph=True`. Choose one or the other.
- [streams](streams.md) are incompatible** with graph.
Comment thread
hughperkins marked this conversation as resolved.
Outdated

### Passing different arguments

Expand Down Expand Up @@ -89,7 +75,7 @@ solve(x, counter)

The argument to `qd.graph_do_while()` must be the name of a scalar `qd.i32` ndarray parameter. The loop body repeats while this value is non-zero.

- On CUDA SM 9.0+ (Hopper), this uses CUDA conditional while nodes — the entire iteration runs on the GPU with no host involvement.
- On [CUDA SM 9.0+](https://developer.nvidia.com/cuda/gpus), this uses CUDA conditional while nodes — the entire iteration runs on the GPU with no host involvement.
- On older CUDA GPUs, AMDGPU, and non-GPU backends, it falls back to a host-side do-while loop (see the [backend support table](#backend-support)).

### Patterns
Expand Down Expand Up @@ -165,7 +151,7 @@ Note that `qd.func`'s are inlined, so you can freely factorize these structures

### Caveats

On platforms without native device-side conditional graph nodes — currently CUDA pre-SM 9.0 and **AMDGPU** (HIP has no conditional / while node API as of ROCm 7.2) — the value of the `graph_do_while` parameter will be copied from the GPU to the host each iteration, in order to check whether we should continue iterating. This causes a GPU pipeline stall. For nested loops this host round-trip happens once per iteration of each loop level, and each loop-body task is replayed individually, so deeply nested loops on these backends pay correspondingly more host overhead (they remain correct, just slower than the CUDA SM 9.0+ native path). At the end of each loop iteration:
On platforms without native device-side conditional graph nodes — currently CUDA pre-SM 9.0 and **AMDGPU** ([HIP](https://rocm.docs.amd.com/projects/HIP/en/latest/what_is_hip.html) has no conditional / while node API as of [ROCm](https://www.amd.com/en/products/software/rocm.html) 7.2) — the value of the `graph_do_while` parameter will be copied from the GPU to the host each iteration, in order to check whether we should continue iterating. This causes a GPU pipeline stall. For nested loops this host round-trip happens once per iteration of each loop level, and each loop-body task is replayed individually, so deeply nested loops on these backends pay correspondingly more host overhead (they remain correct, just slower than the CUDA SM 9.0+ native path). At the end of each loop iteration:

@duburcqa duburcqa Jul 1, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

— currently CUDA pre-SM 9.0 and AMDGPU (HIP has no conditional / while node API as of ROCm 7.2) —

Just cross-reference 'Backend support' section. It is good enough.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would fail doc check, because it's a forward reference.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I said, I think forward reference which are not necessary to read to understand what's going on should be considered fine. It should be reframed as (for details about the supported backend, see 'Backend support' section). At this point, we don't care about which backend is supported or not in practice.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On platforms without [...]

On GPU backends [...] ?

- wait for GPU async queue to finish processing
- copy condition value to hostside
- evaluate condition value on hostside
Expand All @@ -179,7 +165,7 @@ Therefore on unsupported platforms, you might consider creating a second impleme

## Checkpoints with `qd.checkpoint` *(experimental)*

> **Experimental.** `qd.checkpoint`, `qd.GraphStatus`, and `kernel.resume(from_checkpoint=...)` are experimental APIs. The shape of the public surface (the context-manager signature, the `@qd.kernel(checkpoints=True)` flag, the `GraphStatus` fields, the host-side resume loop, the error messages, and the cross-backend lowering details) may change in any future release without a deprecation cycle.
> **Experimental.** `qd.checkpoint`, `qd.GraphStatus`, and `kernel.resume(from_checkpoint=...)` are experimental APIs, and may change in the near future, or be removed, or replaced.

`qd.checkpoint` lets a graph kernel break partway through, surface a reason to the host, let the host fix things up, and resume from the same location on the next launch. An example use-case is an algorithm implemented as a graph that may need to allocate additional memory partway through, where the operations in the graph are in-place, and therefore cannot be rerun without changing/corrupting the output, and therefore for which simply retrying the whole graph from the start is not an option.

Expand Down Expand Up @@ -268,7 +254,21 @@ while status.yielded:
arr[i] = arr[i] + 1
```

The restriction is by design: each top-level statement inside a checkpoint becomes its own GPU task / graph node, so silently wrapping bare statements would hide a sequence of N field writes ballooning into N kernel launches. Forcing the user to write the `for`-wrap themselves keeps the lowering visible and gives a single obvious place to fuse multiple writes into one task by sharing a single wrapper.
The restriction is by design: each top-level statement inside a checkpoint becomes its own GPU task / graph node, so silently wrapping bare statements would hide a sequence of N field writes ballooning into N kernel launches.

## Backend support

`graph=True` and `graph_do_while` run on every backend. They are *hardware accelerated* on CUDA (via CUDA graphs) and AMDGPU (via HIP graphs); `graph_do_while` additionally requires [CUDA SM 9.0+](https://developer.nvidia.com/cuda/gpus) for its hardware-accelerated path. On other backends, `graph=True` is silently ignored and the kernel runs via the normal launch path, and `graph_do_while` falls back to a host-side do-while loop. `qd.checkpoint` gating runs entirely on the device on every GPU backend.

| Feature | `qd.cuda` SM 9.0+ | `qd.cuda` < SM 9.0 | `qd.amdgpu` | `qd.metal` | `qd.vulkan` | `qd.cpu` |
| --- | --- | --- | --- | --- | --- | --- |
| `graph=True` | hardware accelerated | hardware accelerated | hardware accelerated | runs (no acceleration) | runs (no acceleration) | runs (no acceleration) |
| `qd.graph_do_while` | hardware accelerated | host fallback | host fallback | host fallback | host fallback | host fallback |
| `qd.checkpoint` | GPU-side | GPU-side | GPU-side | GPU-side | GPU-side | host-side |

AMDGPU `graph_do_while` falls back to the host-side loop because HIP does not currently expose conditional / while graph nodes (as of ROCm 7.2).

Nested and sibling `graph_do_while` loops (and mixing `graph_do_while` with top-level `for`-loops) are **experimental** for now — see [Nested loops and mixing with for-loops](#nested-loops-and-mixing-with-for-loops).

## Performance

Expand Down Expand Up @@ -393,11 +393,9 @@ k1(a, count)

The recommendation is to use the graph do while here anyway, if you need it for any platform, in order to ensure the code is compact and maintainable.

If you do want fixed-size for loops to run optimally on unsupported hardware platforms, we could add a specializd `qd.graph_range_for` function. This would:
- on graph-do-while-supported hardware: handle adding the additional increment kernel
- on graph-do-while-unsupported hardware: handle running the loop entirely on the host-side, to avoid adding a gpu pipeline stall
If you do want fixed-size for loops to run optimally on unsupported hardware platforms, please raise an issue, and we can look into this.

In practice, for our own kernels, i.e. in genesis-world, they largely fall under the do while formulation, see the previous section. However, also have some that used to be do while, but have been migrated to an optimized fixed-size, see next section.
In practice, for our own [genesis-world](https://github.com/Genesis-Embodied-AI/genesis-world) kernels, they largely fall under the do while formulation, see the previous section. However, also have some that used to be do while, but have been migrated to an optimized fixed-size, see next section.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the point of this sentence? Like, what Quadrants' users are supposed to do with this information?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replace "for our own genesis-world kernels" with "for many real-world cases"?


### A while loop, conditional on a device-side scalar tensor, that has been optimized into a fixed-size for loop

Expand Down