Skip to content

feature: surface aged, suppressed and override counts in the run summary - #12

Open
addyCooks wants to merge 1 commit into
Nano-Collective:mainfrom
addyCooks:feat/surface-filing-counters
Open

feature: surface aged, suppressed and override counts in the run summary#12
addyCooks wants to merge 1 commit into
Nano-Collective:mainfrom
addyCooks:feat/surface-filing-counters

Conversation

@addyCooks

Copy link
Copy Markdown

Closes #11

Description

ReconcileResult already carried incremented, suppressed, and
suppressedByOverride, but the CLI and the persisted run record dropped them
so a pack author calibrating suppressions could not tell whether their
sentinel:false-positive markers were silencing the right findings, or whether
suppression was over-firing.

The per-repo live-run line. Before:

myorg/myrepo: filed 3, touched 5, resolved 0

After:

myorg/myrepo: filed 3, touched 5, aged 2, suppressed 1, suppressed-by-override 0, resolved 0

Default-on and always printed, including zeros per the issue, a zero is the
signal that a layer is not firing, so hiding it hides the calibration answer.
incremented surfaces as aged, which reads better beside resolved.

The run record. FilingSummary carries the same three numbers, summed
across repos in the loop buildRunRecord already runs.

The dashboard. The issue asks for the record to show the full picture in
the static dashboard
, and dashboard.ts rendered only filed so the
counters were persisted but invisible to the org admin triaging a scheduled run.
Added six columns (Filed · Touched · Aged · Suppressed · By override · Resolved), driven off one FILING_COLUMNS table so the header and the row
cannot drift apart. Non-live runs dash out every filing column, as filed
already did.

Formatting moved out of cli.ts into renderFilingLine in
source/run/report.ts. cli.ts has no spec and is excluded from coverage, so
the line was otherwise untestable; this puts the format itself under regression
cover.

No new dependencies, no new files, no hosted surface.

Type of change

  • Bug fix
  • New feature
  • Modification to existing behaviour
  • Documentation
  • Refactor / chore

Testing

Verified end-to-end by driving one real reconcileFindings call into all
three surfaces, so the CLI line, the record, and the dashboard cannot silently
disagree:

CLI       : myorg/myrepo: filed 3, touched 1, aged 1, suppressed 1, suppressed-by-override 1, resolved 1
record    : {"filed":3,"touched":1,"incremented":1,"suppressedByOverride":1,...}
dashboard : 3 | 1 | 1 | 1 | 1 | 1

Each counter is reached via a distinct real path in that fixture: 3 new
findings, 1 hash match, 1 aged misses 0→1, 1 sentinel:false-positive close,
1 vendor/** per-repo override, and 1 resolved at misses 2→3 against
resolveAfterMisses: 3.

That fixture also surfaced an edge case worth guarding: a below-threshold
finding must not inflate suppressed
. It is filtered before planning so it
counts in neither suppression bucket, but nothing asserted it and a regression
there would directly mislead the pack author this feature exists to serve. Added
to the existing threshold test.

11 tests added (253 → 264), 2 rewritten. Edge cases covered:

  • the exact format string from the issue

  • all-zero rendering, proving default-on rather than omit-when-zero

  • cross-repo summation with asymmetric values, so a mis-wired field cannot pass

  • a live run that reconciled nothing

  • filed counting created.length rather than a scalar

  • the audited repo vs targetRepo under aggregate_to_config_repo

  • tolerated errors staying off the line

  • dashboard dash-out on a dry-run, and 0 rendering as 0 (not or ·)

  • a row-cells-equals-column-headings invariant that catches a broken table

  • pnpm test:all passes locally.

  • Added or updated tests for the change.

  • Manual testing notes: run on a clean LF checkout 273 tests pass, coverage
    96.23% lines (dashboard.ts 100%, record.ts 100%, report.ts 99.05%).
    Format, types, lint, knip and audit all pass. Semgrep was not installed
    locally, so test.sh skipped that step; it will run in CI.

Checklist

  • Follows the existing code style (Biome-formatted).
  • I have self-reviewed the diff.
  • Docs updated if user-visible behaviour changed.
  • No secrets, tokens, or private source committed.
  • Contract surfaces (sentinel.yaml schema, rule-pack manifest, findings
    model) if changed, the docs and the relevant spec are updated in this PR.
  • Breaking changes are called out above.

Notes for review

  • FilingSummary gains three required fields. It is exported from
    source/index.ts, so anyone constructing one by hand against the library API
    must add them. Records already written stay readable the dashboard renders
    a missing counter the same way it renders a dry-run. Flagging it as the one
    contract-ish surface this touches; happy to make the fields optional if you
    would rather it be additive-only.
  • Docs: added a note to the Suppression section of docs/findings/index.md,
    since the three layers documented there map exactly onto touched /
    suppressed / suppressed-by-override, which is the calibration emphasis
    the issue points at.
  • The dashboard columns are the "your call" half of my comment on the issue
    I read the proposed solution as asking for them ("shows the full picture in
    the static dashboard"). Happy to split them into a follow-up if you would
    rather keep this PR to the CLI and the record.

…n summary

ReconcileResult already carried incremented, suppressed and
suppressedByOverride, but the CLI and the persisted run record dropped
them, so pack authors calibrating suppressions could not tell whether
their markers were silencing the right findings.

The per-repo live-run line now reads:

  myorg/myrepo: filed 3, touched 5, aged 2, suppressed 1, suppressed-by-override 0, resolved 0

FilingSummary carries the same three numbers, so the run record
committed to the config repo shows the full picture in the dashboard.

Formatting moves into renderFilingLine so the line is covered by tests
rather than living untested in cli.ts.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Surface incremented / suppressed / suppressedByOverride in the run summary

1 participant