Skip to content

fix(audit): exempt test code from layer-2 CRITICAL-pattern gate#45

Merged
XuebinMa merged 1 commit into
mainfrom
worktree-audit-gate-test-exempt
Jun 3, 2026
Merged

fix(audit): exempt test code from layer-2 CRITICAL-pattern gate#45
XuebinMa merged 1 commit into
mainfrom
worktree-audit-gate-test-exempt

Conversation

@XuebinMa
Copy link
Copy Markdown
Owner

@XuebinMa XuebinMa commented Jun 3, 2026

What

Exempt test code from the Layer-2 pre-commit audit gate's CRITICAL-pattern
scan (.expect() / panic!() / .unwrap() / unsafe / shell-out).

Why — observation-period unfreeze

The three-layer audit workflow ran frozen (C-layer) from 2026-05-08 to
2026-06-01. Triaging the accumulated Layer-1 findings (.audit-state/findings.log,
7 CRITICAL hits):

Disposition Count Detail
False positive — test code 3 .expect() / panic!() in tests.rs, an inline #[cfg(test)] mod helper, and a test assertion
Remediated prod .expect() 3 secrets.rs ×2 → Regex::new(pat).ok(); provenance.rs → removed
True positive, justified 1 bash.rs from_utf8(...).expect(...) with a documented SAFETY invariant

Signal: the gate caught 3 real prod .expect()s (all since remediated) —
it works. Its only false-positive class is test code, which is idiomatic and
never executes on the host.

How

The gate diffs with -U0, so the enclosing #[cfg(test)] attribute isn't in
the diff. The scanner now reads each flagged file and brace-matches to
compute test-scope line ranges:

  • Whole test files exempted: tests.rs, **/tests/**, *_test.rs.
  • Inline exempted: lines inside a #[cfg(test)] module or a #[test] /
    #[<runtime>::test] function.
  • Comments and double-quoted strings are stripped before counting braces, so
    "} {" in a string literal doesn't skew depth.
  • Unbalanced blocks are not exempted — bias toward flagging, never toward
    silently letting a real prod .expect() through.

Verification

Ran the actual scanner block (extracted from the script, no copy) against a
synthetic fixture mixing prod and test .expect()s plus a brace-bearing string:

  • Prod .expect() flagged (lines 2, 16) — including one placed immediately
    after a test module, proving the range doesn't over-extend.
  • Test-scope .expect() exempted (lines 8 inside #[cfg(test)] mod, 13 inside
    #[test] fn). PASS.
  • bash -n clean.

Decisions

  • LLM-dispatch kept stubbed. Deterministic gate + test exemption gives a
    high enough signal-to-noise ratio; the single justified-but-flagged prod case
    doesn't justify synchronous per-commit LLM cost. Revisit if such cases grow.
  • Full triage recorded locally in docs/audits/observed-fp.md (gitignored by
    design — that dir is local Layer-3 observation state).

Scope

Only .claude/workflows/pre-commit-review.sh. No Rust source, no audit-rules.yaml
or hook-registration changes.

Observation-period unfreeze (2026-05-08 → 2026-06-01). 3 of 7 Layer-1
findings were test-code false positives: .expect()/panic!()/.unwrap() are
idiomatic in tests and never run on the host. The gate's -U0 diff carries
no enclosing-attribute context, so the scanner now reads each flagged file
and brace-matches #[cfg(test)] modules and #[test]/#[<rt>::test] functions
to compute test-scope line ranges; whole test files (tests.rs, tests/,
*_test.rs) are exempted outright. Strings/comments are stripped before
brace counting; unbalanced blocks are not exempted (bias toward flagging).

Records the full observation-period triage in docs/audits/observed-fp.md
(3 prod expects already remediated to .ok()/removed; 1 justified prod
expect; LLM-dispatch kept stubbed).
@XuebinMa XuebinMa merged commit 6bcf6bf into main Jun 3, 2026
12 of 14 checks passed
@XuebinMa XuebinMa deleted the worktree-audit-gate-test-exempt branch June 3, 2026 05:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants