Skip to content

ci: impute a weight for new PostgreSQL tests instead of refusing them - #821

Merged
jason931225 merged 1 commit into
mainfrom
ci/impute-unmeasured-postgres-weights
Aug 19, 2026
Merged

ci: impute a weight for new PostgreSQL tests instead of refusing them#821
jason931225 merged 1 commit into
mainfrom
ci/impute-unmeasured-postgres-weights

Conversation

@jason931225

Copy link
Copy Markdown
Collaborator

The defect

#815 packs the PostgreSQL shards by measured duration and fails closed on any workflow entry with no measured_seconds. But weights are harvested from cargo-postgres-timing: lines in a finished CI log — so a test that has never run cannot have a measurement, and it cannot run until its PR is green.

Every new PostgreSQL test is currently unmergeable. #802 is the first to hit it:

1 workflow entr(ies) have no measured_seconds; regenerate weights before trusting the balance:
  ontology-canonical-adapter-postgres-employment-reassign-as-runtime-role-pg

Why the guard was still right, and what changes

That guard was written for a different failure — a map that has drifted, where entries silently lost weights they once had. Both look identical to a per-entry check. They differ by scale: a PR adds one or two, a bad regeneration drops dozens. So the distinction is now made on share.

case before after
one new test ❌ blocked imputed + named, exit 0
>10% unmeasured (drift) ❌ blocked ❌ still blocked
no measurement anywhere ❌ blocked ❌ blocked, distinct message
  • Unmeasured entries are imputed at the mean of their own package where it has a measured sibling (a new suite usually resembles the suite it joins), global mean otherwise.
  • Never 0. A free entry lands in the lightest bin — exactly where a heavy newcomer hurts most — and treating unmeasured as free is how the entry-count scheme drifted in the first place.
  • Imputation bases are computed over measured entries only, so an imputed value never feeds back into the mean used to weigh the next one.
  • The CLI names every imputed entry and the seconds it added, so an estimate stays visible and gets replaced at the next harvest rather than hardening into a fake measurement.

Measured on #802's exact case

imputed 35.4s across 1 unmeasured entry (replace at the next timings harvest):
  ontology-canonical-adapter-postgres-employment-reassign-as-runtime-role-pg
partition ok (5 shards, max 668.3s, spread 1.00x)

Mutation-proven on the real 209-entry map

mutation result
strip 15% of measurements exit 1 — "14.8%, limit 10%; the map has drifted"
strip every measurement exit 1 — "no basis to impute from"
one new entry (#802's shape) exit 0 — imputed and named

26 partition tests pass; 12 gates swept, 0 failed; cargo-map, dark-suites and executed-tests-baseline all exit 0.

Unblocks

#802, and every future PostgreSQL test.

🤖 Generated with Claude Code

#815 packs the PostgreSQL shards by measured duration and fails closed on
any workflow entry with no `measured_seconds`. Weights are harvested from
`cargo-postgres-timing:` lines in a FINISHED CI log, so a test that has
never run cannot have a measurement -- and it cannot run until its PR is
green. The guard therefore made every new PostgreSQL test unmergeable.
#802 is the first one to hit it:

  1 workflow entr(ies) have no measured_seconds; regenerate weights
  before trusting the balance:
    ontology-canonical-adapter-postgres-employment-reassign-as-runtime-role-pg

That guard was written for a different failure: a map that has DRIFTED,
where entries silently lost weights they once had. Both look identical to
a per-entry check, but they differ by scale -- a PR adds one or two, a bad
regeneration drops dozens -- so the distinction is now made on share.

  - Unmeasured entries are imputed at the mean of their OWN package where
    it has a measured sibling (a new suite usually resembles the suite it
    joins), and the global mean otherwise. Never 0: a free entry lands in
    the lightest bin, which is exactly where a heavy newcomer hurts most,
    and treating unmeasured as free is how the entry-count scheme drifted
    in the first place.
  - Above MAX_UNMEASURED_SHARE (10%) the map is treated as drifted and
    still fails closed. At the observed 209 entries that allows 20 -- far
    more than any PR adds, far fewer than a regeneration that lost its
    timings.
  - A map with NO measurement anywhere fails closed separately: there is
    no basis to impute from, and imputing from nothing would invent a
    balance nobody measured.
  - Imputation bases are computed over measured entries only, so an
    imputed value never feeds back into the mean used to weigh the next
    one.
  - The CLI names every imputed entry and the total seconds it added, so
    an estimate is visible and gets replaced at the next timings harvest
    rather than hardening into a fake measurement.

Measured on #802's exact case: imputed 35.4s from its package mean,
partition ok, 5 shards, spread 1.00x, exit 0.

Mutation-proven on the real 209-entry map:
  strip 15% of measurements -> exit 1, "14.8%, limit 10%, the map has
    drifted";
  strip every measurement   -> exit 1, "no basis to impute from";
  one new entry (the #802 shape) -> exit 0, imputed and named.

26 partition tests pass; 12 gates swept, 0 failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jason931225
jason931225 added this pull request to the merge queue Aug 19, 2026
Merged via the queue into main with commit 61bd9c0 Aug 19, 2026
32 checks passed
@jason931225
jason931225 deleted the ci/impute-unmeasured-postgres-weights branch August 19, 2026 06:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant