Repository navigation
Description
From the gate evaluation, kb/designs/gate-evaluation-2026-11.md (#1638), "The three changes", ticket 2. Filed at the maintainer's word (2026-11-17, the efficiency plan); written so a contributor can start from this issue alone.
Groom 2026-11-17
Automated groom (an AI agent working for the maintainer), copied from the evaluation as merged on dev 0eb1bcf. The doc's tables and grep commands are the evidence.
Observation. Ten red PR runs came from source-reading tests. Seven were
loop PRs, whose authors run the pre-push hook. Reproduced:
scripts/test-affected --list --files extensions/software-kb/src/pyrite_software_kb/board.py
selects 86 files and none of test_kb_reads_are_one_function,
test_link_reads_pass_a_scope, test_response_text_is_pyrites. Those are
the three that failed on #1553 in CI (runs 43985141356, 43988113479). For
pyrite/services/qa_service.py it misses three of the twelve checked.
Principle 8. The selection is the one place the rule "run what your
change can break" lives. Today the gap is covered by a row in the worker
skill (#1554), which is a rule each worker must remember, and outside
authors never read it.
User's model. "If my push passes locally, CI will not surprise me with
a rule about the source." CONTRIBUTING already promises this for guards.
Implementation model. Import reach cannot see a test that reads files.
A marker says which tree a test reads; a changed path under that tree
selects it.
Acceptance: A change to any file under pyrite/, extensions/*/src/ or scripts/
selects every test function that scans that tree. `--explain`
says so. A test function that scans a source tree and carries no
marker fails a completeness test. Fails today: a parametrised
test over (changed file, scanning test) pairs, starting with the
three above.
Regimes: a deleted source file; a change only under tests/ (guards, as
today); a full-suite trigger (unchanged); the stamp must still
cover the selection.
Shapes: 56 files call ast.parse, rglob or inspect.getsource (grep below).
Classify every scanning function: marked | not a scanner (the
call parses test input) | out of scope and why. Two layouts: a
file that is all scanners (test_layer_boundaries.py) and a
behaviour file with one scanner (test_index_schema_drift.py: 39
tests). Mark functions, not modules, as `guard` does. Trees read:
pyrite/, extensions/*/src, scripts/, docs/, tests/.
Touches: existing: scripts/test-affected, tests/test_test_affected.py,
pyproject.toml (marker), the scanning test files (one decorator
each), CONTRIBUTING.md (Guards), .claude/skills/pyrite-dev
(the row becomes a pointer) new: none
Sequence: independent of ticket 1. After #1614 if both edit
tests/test_cli_numeric_options_declare_a_range.py.
Model: sonnet
heavy: no
Cold read: no (it only adds tests to a selection)
Out of scope: selection by execution reach (tests-q4). Registry-walking tests
that import what they walk; import reach already finds them
(checked: test_cli_numeric_options and
test_every_entry_point_passes_the_policy are selected for a
pyrite/ change).
Open question for the worker: one marker with an argument
(guard(reads="pyrite")) or a second marker. Predicted footprint: about 40
files, most of them one line; scripts/test-affected about 60 lines.
Changing a script widens that PR's matrix to three interpreters.
Related: #1554 (the worker-skill row this replaces with a pointer), #1553 (the PR whose CI runs showed the gap), #1614 (shares one test file).
Activity