ground-truth-gates item 3: a substring grader must key on a token its corpus cannot produce

[firaen22]

What

One fake-pass shape added to ground-truth-gates §item 3, plus a Provenance
entry.

A substring grader that scans prose for a match token the graded corpus can
itself contain scores the corpus, not the behavior.

Why it isn't covered by the shapes already in item 3

The existing entries — empty glob, moved directory, missing baseline, scanner
that matched zero inputs — all share a signature: the gate ran over nothing.
The prescribed fix follows from that (assert the matched count is non-zero).

This one runs correctly, over a non-empty input, and still passes every arm.
The non-empty assertion does not catch it.

Its failure direction is also inverted from the rest of the list. A vacuous
green hides a failure that exists. This one manufactures agreement: in an
A/B, every arm passes, the run reads as "no effect", and a rule that genuinely
discriminated gets retired on the strength of a grader bug. A false negative
about your own experiment is harder to notice than a false green, because the
null result is the outcome you were already prepared to accept.

Fix is two-part, both cheap:

• key on a structural marker the subject must create (a heading, a filename,
  a field), not a word it might merely mention;
• sanity-check the grader against a known-bad arm before trusting a clean sweep
  — a grader that passes an arm you know failed is the finding, not a formality.

Provenance

One downstream consumer's A/B probe. The grader for a "did the arm record the
change?" axis scanned each arm's output file for the word drift — which the
fixture's own body text already contained ("a batch that drifts model mid-run").
All four arms passed, including three that recorded nothing. The run read as a
4/4 null result. Regraded on a port-note heading, the same arms split 0/3 bare
vs 1/1 ruled, and the rule under test was real.

Ships unprobed per the covenant.

Verification

• Scoped: git diff --stat touches skills/ground-truth-gates/SKILL.md only.
• Additive: word-diff shows zero removed words (the 1 deletion is a split line).
• All new lines ≤ 80 chars by character count.
• Non-ASCII limited to the house em-dash.
• Branched fresh off upstream/main (a6ff7d0); staged by explicit pathspec.

[F-e-u-e-r]

Evaluation record (maintainer). Merged via combined PR #90 (a148180), after an 11-round cross-model gate (grok-4.5 high + gpt-5.6-luna ultra every round, gpt-5.6-sol max from r3).

The fifth fake-pass shape was upheld as a genuine new class (manufactured agreement over real input — the inverted failure direction argument held up). Gate folds applied:

• The list lead-in corrected "Four more" → "Five more" (your insertion made the count false — caught in round 1 by grok).
• "unlike the empty-scan shapes above" → "unlike the zero-input scanner above" (the preceding list also holds non-empty-scan shapes; both first-round lenses converged on this).
• The marker must be created AND filled (an empty-emitted heading could still false-pass a content axis).
• The damage line no longer claims uniqueness ("Its damage is not a vacuous empty run — it manufactures agreement over real input") — a zero-input A/B can also all-pass.
• Provenance labeled contributor-reported/not-linkable.

One reviewer escalation was rejected with counter-evidence in your favor: a demanded malformed-marker negative-control apparatus — the entry's adjacent sentence already binds item 2's known-broken-run discipline over this grader class ("no exemption"), and deeper grader validation is items 1–2 territory.

Final decision: accepted and merged with the folds above; ships unprobed, joins the round-5 queue.