compute decouple

.github/workflows/bench-regress.yml

@@ -27,7 +27,7 @@ on:
 jobs:
   bench:
     # macos-14 = Apple Silicon (M1+). Required for the metal cells —
-    # without it, drop --features metal from FEATURES to skip them
+    # without it, drop --features gpu from FEATURES to skip them
     # and run only the CPU surface on any runner.
     runs-on: macos-14
     timeout-minutes: 90

.github/workflows/larql-cli.yml

@@ -101,7 +101,7 @@ jobs:
         if: runner.os != 'macOS'
         run: cargo check -p larql-cli --bins --tests --no-default-features
 
-      - name: Check (default features incl. metal) on macOS
+      - name: Check (default features incl. gpu) on macOS
         if: runner.os == 'macOS'
         run: cargo check -p larql-cli --bins --tests
 

AGENTS.md

@@ -14,14 +14,36 @@ Cargo workspace at repo root with a strict dependency chain — respect this whe
 
 ```
 # LARQL-specific (depend on vindex, LQL, etc.)
-larql-models      model config, architecture traits, weight loading, quant/dequant
+larql-models      model config, architecture traits, weight loading, quant/dequant,
+                  shared test_fixtures (behind `test-utils` feature)
     ↓
-larql-compute     CPU/Metal matmul backends, pipeline
+larql-compute     CPU substrate: BLAS kernels, residual norms, attention spine
+                  (rope/gqa/block/decode/gpu), forward-pass primitives (embed,
+                  ops, hooks, ple, layer, predict/raw), kquant_forward Q4_K/Q6_K
+                  decode helpers, FfnBackend trait + dense WeightFfn impl,
+                  KvDispatch + AsyncComputeBackend traits + CpuBackend impls,
+                  KvIndex trait (abstracts VectorIndex for substrate callers),
+                  forward_overrides env-var registry, PerLayerDecodeState.
+                  ADR-0022 moved all of this down from larql-inference; the
+                  substrate is now self-contained.
     ↓
-larql-vindex      vindex lifecycle: extract, load, query, mutate, patch, save, Vindexfile
+larql-compute-metal  Metal GPU backend (first-class peer, NOT a thin layer).
+                     Implements ComputeBackend / KvDispatch / AsyncComputeBackend
+                     for MetalBackend; ships custom MSL shaders, multi-layer
+                     pipelining, stage-bisected kernels.
+    ↓
+larql-vindex      vindex lifecycle: extract, load, query, mutate, patch, save,
+                  Vindexfile. Implements `KvIndex for VectorIndex` (Step 3a).
     ↓
 larql-core        graph algorithms (merge, diff, BFS, pagerank, shortest-path)
-larql-inference   forward pass, BLAS-fused attention, Metal GPU, WalkFfn, trace
+larql-inference   engines (Standard, MarkovResidual, Apollo, etc.), chat,
+                  sessions, tokenizer, FFN routing impls (Graph/Remote/MoE),
+                  layer_executor, layer_graph orchestration. Substrate moves
+                  to larql-compute; this crate is the inference-shaped layer
+                  that composes substrate primitives + engine state. Re-export
+                  shims preserve `crate::{residual, forward, attention,
+                  kv_dispatch, async_compute_backend, kquant_forward,
+                  forward_overrides}::*` paths for back-compat.
     ↓
 larql-lql         lexer/parser/executor/REPL + USE REMOTE client
     ↓
@@ -34,6 +56,15 @@ model-compute         bounded native kernels (arithmetic/datetime) and optional
                       wasmtime-hosted WASM modules (features: `native`/`wasm`)
 ```
 
+**Metal is a first-class peer** (ADR-0022, 2026-05-18). `larql-compute-metal`
+is the same shape as a future `larql-compute-vulkan` / `larql-compute-cuda` —
+its own crate, implements the same trait surface, owns its kernels. Inference
+factories (`default_engine_backend()`, `default_async_engine_backend()`,
+`default_compute_backend()` in `larql-inference/src/lib.rs`) compose Metal +
+CPU fallback explicitly; engine-level orchestration in `layer_graph/` still
+branches on `#[cfg(feature = "gpu", target_os = "macos")]` where the
+hybrid + GPU prefill paths take backend-specific actions.
+
 **`model-compute` never imports `larql-*`.** Dependency flow is one-way:
 LARQL may consume it (e.g. for compile-time `sum(1..100)` resolution); it
 knows nothing about vindex or LQL. When it moves to a sibling repo, the
@@ -51,10 +82,10 @@ LQL parser and executor are split symmetrically: [crates/larql-lql/src/parser/](
 
 ```bash
 cargo build --release                             # optimised build
-cargo build --release --features metal            # Metal GPU backend (Apple Silicon)
+cargo build --release --features gpu              # GPU backend (Metal today; Vulkan/CUDA later)
 cargo test                                        # entire workspace
 cargo test -p larql-lql                           # single crate (272 tests)
-cargo test -p larql-inference --features metal    # +Metal GPU tests
+cargo test -p larql-inference --features gpu      # +GPU tests (Metal on Apple Silicon)
 cargo test -p <crate> <test_name>                 # single test
 make ci                                           # fmt-check + clippy -D warnings + test
 make fmt                                          # cargo fmt --all
@@ -82,6 +113,8 @@ Or via the Makefile: `make python-setup | python-build | python-test | python-cl
 - **Three extraction levels, not features.** `browse` (~3 GB), `inference` (~6 GB), `all` (~10 GB) — gated by `ExtractLevel` enum in [crates/larql-vindex/src/config/types.rs](crates/larql-vindex/src/config/types.rs). Check level before attempting an operation; fail loudly if weights aren't present.
 - **Walk FFN is sparse-by-design and can beat dense** (517ms vs 535ms on Gemma 4B) because gate KNN (K≈10) skips most of the 10,240 features per layer. If you touch FFN code, preserve this invariant — see [docs/ffn-graph-layer.md](docs/ffn-graph-layer.md).
 - **MXFP4 quantized MoE (GPT-OSS) has degraded DESCRIBE/WALK** due to 4-bit precision; `INFER` is the supported path. Don't assume all model families are equivalent — see [docs/specs/vindex-operations-spec.md](docs/specs/vindex-operations-spec.md).
+- **Substrate-vs-engine split** (ADR-0022): all CPU forward-pass math + attention + KvDispatch/AsyncComputeBackend traits live in `larql-compute`, not `larql-inference`. When adding a new substrate primitive (a kernel, an attention variant, a new norm), put it in `larql-compute` and re-export from `larql-inference` for back-compat. When adding engine-shaped code (a new session type, an FFN routing impl, a layer-graph dispatcher), it stays in `larql-inference`. The rule of thumb: substrate consumes `&dyn larql_compute::KvIndex` + `ModelWeights`; engines consume sessions, tokenizers, gRPC clients, layer_graphs.
+- **VectorIndex is reached through `KvIndex` from substrate.** `larql-compute`'s `KvDispatch` + `AsyncComputeBackend` + `kquant_forward` take `Option<&dyn KvIndex>` parameters. `larql-vindex` impls `KvIndex for VectorIndex` in `kv_index_impl.rs`. Engine callers passing `&VectorIndex` to substrate traits coerce with `.map(|v| v as &dyn larql_compute::KvIndex)`. Don't reach for `larql_vindex::*` from inside `larql-compute` — that's the cycle the trait was created to avoid.
 
 ## Where to find things
 

Cargo.toml

@@ -64,3 +64,10 @@ debug = "line-tables-only"
 
 [profile.test.package."*"]
 debug = false
+
+# Temporary: enable line tables in release builds so samply / flamegraph
+# can resolve symbol names. Cheap (no full debug info; just symbols
+# + line numbers). Remove once profiling is done.
+[profile.release]
+debug = "line-tables-only"
+split-debuginfo = "packed"

Makefile

@@ -91,9 +91,12 @@ larql-models-bench-test:
 
 # larql-models - architecture detection, weight loading, quant codecs.
 #
-# Per-file 90% floor, total at floor(current) - see
-# crates/larql-models/coverage-policy.json for current debt baselines.
-LARQL_MODELS_COVERAGE_MIN ?= 94
+# Per-file 90% floor; whole-crate total at 80 since `cargo llvm-cov` includes
+# the `test_fixtures.rs` support file (test-utils feature, ~30% covered when
+# measured here in isolation — see crates/larql-models/coverage-policy.json
+# for the full reasoning). The real 94% bar is enforced by the policy
+# script's `included_total_line_min_percent` over the non-fixture files.
+LARQL_MODELS_COVERAGE_MIN ?= 80
 LARQL_MODELS_COVERAGE_POLICY ?= crates/larql-models/coverage-policy.json
 LARQL_MODELS_COVERAGE_REPORT ?= coverage/larql-models/summary.json
 

README.md

@@ -744,7 +744,7 @@ The full surface is documented in `crates/larql-inference/ROADMAP.md` §
 
 | Platform | Compiles | GPU | BLAS |
 |----------|----------|-----|------|
-| macOS arm64 (M-series) | ✓ | Metal (`--features metal`) | Accelerate |
+| macOS arm64 (M-series) | ✓ | Metal (`--features gpu`) | Accelerate |
 | Linux arm64 / x86_64 | ✓ | — (CPU fallback) | OpenBLAS |
 | Windows arm64 / x86_64 | ✓ | — (CPU fallback) | OpenBLAS |
 
@@ -754,11 +754,11 @@ macOS gets Metal GPU acceleration. Linux and Windows run the same CPU path (BLAS
 
 ```bash
 cargo build --release                    # optimised build
-cargo build --release --features metal   # with Metal GPU backend (macOS only)
+cargo build --release --features gpu     # with GPU backend (Metal on macOS today; Vulkan/CUDA later)
 cargo test                               # all tests across all crates
 .venv/bin/python scripts/diagnose_models.py    # cross-engine correctness sweep — see below
 cargo test -p larql-inference            # inference engine tests (109 tests)
-cargo test -p larql-inference --features metal  # + Metal GPU tests (115 tests)
+cargo test -p larql-inference --features gpu    # + GPU tests (115 tests)
 cargo test -p larql-lql                  # LQL parser + executor tests (272 tests)
 cargo test -p larql-vindex               # vindex storage + patch tests (525 tests as of 2026-05-08)
 
@@ -776,8 +776,8 @@ make larql-vindex-coverage-html          # HTML report plus the same policy gate
 cargo run --release -p larql-inference --example attention_demo    # fused attention demo
 cargo run --release -p larql-inference --example mech_interp_demo  # capture / lens / ablate / steer / patch (synthetic — no vindex)
 cargo run --release -p larql-inference --example bench_attention   # attention benchmarks
-cargo run --release -p larql-inference --example backend_demo --features metal   # backend demo
-cargo run --release -p larql-inference --example bench_backend --features metal  # backend benchmarks
+cargo run --release -p larql-inference --example backend_demo --features gpu   # backend demo
+cargo run --release -p larql-inference --example bench_backend --features gpu  # backend benchmarks
 cargo run --release -p larql-inference --example bench_inference   # full inference benchmarks
 
 # Vindex tools (build once, enables mmap walk)