Skip to content

delegation-and-review §1: a pinned model string does not pin behavior - #54

Merged
F-e-u-e-r merged 2 commits into
F-e-u-e-r:mainfrom
firaen22:mine-pinned-string-behavior-drift
Jul 22, 2026
Merged

delegation-and-review §1: a pinned model string does not pin behavior#54
F-e-u-e-r merged 2 commits into
F-e-u-e-r:mainfrom
firaen22:mine-pinned-string-behavior-drift

Conversation

@firaen22

Copy link
Copy Markdown
Contributor

What

Adds one bullet to delegation-and-review §1, directly extending the existing "Model lineups are volatile facts" sentence: a pinned model string does not pin behavior — a measured behavioral property of a hosted endpoint (an edge-guard rate, a failure signature, a latency class) is a measurement of a day, not of the string. Date-stamp such numbers where recorded; when a routing or safety decision depends on one, re-run the probe instead of reusing it; an undated behavioral claim about a hosted endpoint is expired on arrival.

Why

§1 already covers lineup volatility (which models exist) and cross-model-review §1 covers identity discovery — neither covers the measurements themselves drifting behind an unchanged slug. Two independent private incidents in one week, two vendors: a pre-registered one-day re-measurement of a CLI's unstated-edge guard rate (same --model flag, same prompt battery) flipped enough to force retraction of the prior day's published claim; a second vendor's endpoint inverted a reproduced infinite-loop failure into a fully guarded pass over twelve days, model strings unchanged. The failure the rule targets is reusing a stale benchmark number to route edge-risky work.

Covenant status

Both incidents contributor-reported (private benchmark harnesses, verifiable by the contributor, not linkable); cited as shape, numbers not restated. In-body unprobed marker; Provenance notes that scheduled endpoint drift cannot be fixtured — longitudinal re-measurement is the honest test, recorded as debt.

Checks

  • Single-file diff (+27 lines), no other files touched
  • Unicode sweep of added lines: only house-style §/—/❌
  • python3 .github/checks.py: all checks passed

🤖 Generated with Claude Code

firaen22 and others added 2 commits July 22, 2026 08:24
Two independent private incidents in one week, two vendors: a pre-registered
one-day re-measurement of a CLI's unstated-edge guard rate (same --model
flag, same prompt battery) flipped enough to force retraction of the prior
day's published claim; a second vendor's endpoint inverted a reproduced
infinite-loop failure into a fully guarded pass over twelve days behind
unchanged model strings. §1 already says lineups are volatile; this says the
MEASUREMENTS are too — a behavioral number about a hosted endpoint is a
measurement of a day, not of the string. Date-stamp such numbers, re-run
the probe when a routing/safety decision depends on one, treat undated
claims as expired. Ships unprobed per the README covenant (nothing can
fixture scheduled endpoint drift); Provenance records longitudinal
re-measurement as the honest test.
…view

PR F-e-u-e-r#54 round-1 gate (grok-4.5 high: FIX 1; gpt-5.6-sol max: FIX 1-6).
All reproduced and addressed:
- decision-time re-run made canonical; DAY framing replaced by
  not-a-durable-attribute ontology; fresh results INFORM routing and
  never replace S2's edge spec + proof gate; no measurement pins the
  next request (grok F1 / codex F1)
- 'number' -> 'measurement' covering categorical properties
  (grok n2 / codex F2)
- terminal re-verification line now includes recorded hosted-endpoint
  behavioral claims (codex F3)
- Done defined (fresh probe cited w/ timestamp+config, or the
  unknown-property fallback: route as if unguarded + spec the edge);
  probe-unavailable disposition named; pos example added; neg example
  generalized to any prior measurement incl. this morning's
  (grok F1 / codex F4)
- provenance: rates-not-restated claim qualified (intervals are
  contributor-reported shape); in-body interval phrasing loosened to
  match (codex F5)
- probe debt re-characterized as the behavioral executor trap
  (stale dated result + changed same-slug probe), distinct from
  premise re-verification (codex F6)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GhLu6NCGzM9HxnqNb87MAx
@F-e-u-e-r
F-e-u-e-r merged commit 910e94d into F-e-u-e-r:main Jul 22, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants