From 7e3846985ab96464e993a740833a4cc6bad5f36a Mon Sep 17 00:00:00 2001 From: firaen22 Date: Mon, 27 Jul 2026 23:58:43 +0800 Subject: [PATCH 1/3] =?UTF-8?q?skill-authoring=20=C2=A77:=20bare=20arm=20i?= =?UTF-8?q?s=20environment-relative;=20harvest=20the=20arms'=20reasoning,?= =?UTF-8?q?=20not=20just=20verdicts?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two clauses folded into the existing bare-probe bullet, distilled from a downstream consumer's first four fold probes of this repo's merged rules: - The bare arm runs in the executor's REAL baseline (standing global rules, always-loaded caches), so a bare-pass verdict means redundant in that environment only — record the baseline next to the verdict and never export non-discrimination to a different baseline. - Read each arm's stated reasoning before writing the fold: a failing bare arm supplies the exact excuse the rule must refute (carry it in near-verbatim); a passing ruled arm that over-fires by refusing the surrounding task is a wording warning, not a win. Provenance paragraph appended; both clauses ship unprobed per the covenant. Co-Authored-By: Claude Fable 5 --- skills/skill-authoring/SKILL.md | 42 +++++++++++++++++++++++++++++++-- 1 file changed, 40 insertions(+), 2 deletions(-) diff --git a/skills/skill-authoring/SKILL.md b/skills/skill-authoring/SKILL.md index a7941d1..7e78672 100644 --- a/skills/skill-authoring/SKILL.md +++ b/skills/skill-authoring/SKILL.md @@ -596,7 +596,20 @@ default; an AI rewrite does not launder a derivative). gaps: what the file fails to make happen. This asks the opposite question, before the rule exists — run the scenario twice against the file's target executor, as independent fresh invocations (no shared state or history - between the arms), once with no rule and once with it. An arm counts only + between the arms), once with no rule and once with it. The bare arm is + bare of THIS rule, not of the world: it runs in the executor's real + baseline — standing global rules, always-loaded caches, whatever the + environment ships to every invocation — because that baseline is the + live counterfactual for the fold decision, and stripping it makes the + probe answer a question nobody asked (does the rule beat a blank + model?). Every verdict then inherits the baseline's scope: a bare ✓ + means redundant IN THAT ENVIRONMENT, not redundant everywhere — a + baseline carrying a standing general form of the candidate (a global + verify-before-relaying order) fails the specific form's probe locally + while the same rule still earns its line wherever no such standing + order exists, so record what the baseline contained next to the + verdict, and never export a non-discrimination result to an + environment with a different baseline. An arm counts only when its run demonstrably met the rule's trigger (ground-truth-gates' not-armed discipline: a run that never hit the guarded condition is excluded and re-run), and one run per arm screens for a large effect @@ -610,7 +623,17 @@ default; an AI rewrite does not launder a derivative). arm produces it → the rule earns its line; neither arm does → the rule as written is ineffective — rewrite or drop it, never fold it in on truth alone; only the bare arm produces it → the rule is harmful — dropped, not - filed as reference. Eight rules folded into two files over one week were + filed as reference. The verdicts are not the whole harvest — read each + arm's stated REASONING before writing the fold. A failing bare arm hands + you the exact rationalization the folded rule must refute; carry that + excuse and its rebuttal into the rule line near-verbatim rather than + restating the principle (the excuse is what fires under pressure — the + compression bullet below is why the rebuttal must survive). And a + passing ruled arm can still over-fire, satisfying the rule by refusing + the surrounding task outright; that over-fire is a wording warning, not + a win — reword the fold so compliance composes with doing the job + (verify FIRST, then proceed), or the folded rule trades one failure + mode for another. Eight rules folded into two files over one week were probed this way afterwards; three were reproduced unaided. Non-discriminating is not the same as wrong — the rule can be true and still not worth its line, and that is the judgment the probe is for. @@ -996,3 +1019,18 @@ copied faithfully from a source file carrying the same error, so fixing the derived file alone would have been undone by the next write-back — the read-the-source rule. All four ship `unprobed` per the covenant; their probes join the private round-5 queue. +The §7 bare-probe bullet's environment-relative-baseline and +read-the-arms'-reasoning clauses (2026-07-27) are class-distilled from one +downstream consumer's first fold probes of this repo's own merged rules (four +rules probed, pre-registered arms, weaker-tier executor). Two probes produced +the clauses. A fact-anchoring rule failed to discriminate solely because the +consumer's global rules file carries a standing verify-before-relaying order — +the general form of the candidate — so the local verdict (redundant) was real +there and false anywhere without that baseline: the environment clause. And in +both discriminating probes the arms' reasoning outlived their verdicts: one +failing bare arm defended shipping an unverified capability doc with "the +document didn't lie — it described the intended behavior", which became the +folded rule's named rebuttal, while one passing ruled arm over-fired and +refused to produce any plan at all, forcing a reword so compliance composed +with the task: the reasoning clause. Both clauses ship `unprobed` per the +covenant; their probes join the private round-5 queue. From 16e6e1f91b6dd82201f14eac4adb81a5634fdfa4 Mon Sep 17 00:00:00 2001 From: firaen22 Date: Tue, 28 Jul 2026 08:52:57 +0800 Subject: [PATCH 2/3] =?UTF-8?q?skill-authoring=20=C2=A77:=20separate=20bas?= =?UTF-8?q?eline=20from=20leakage=20in=20the=20bare=20arm?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Third clause on the same bare-probe bullet, and it must ship with the first one: clause 1 says the bare arm KEEPS its environment (standing global rules, always-loaded caches); this says strip episodic recall of the finding under test. Stated apart, each reads as contradicting the other — the boundary is the rule. The test is what the arm would have without you: what every invocation in that environment ships with is baseline; an auto-memory store, a session summary, or notes from the run being probed are leakage. That store survives a fresh invocation, so the existing no-shared-state requirement does not catch it — the arm loads the conclusion, restates it, and scores as an unaided reproduction. Countermeasures: recall-dodging framing, an explicit disregard instruction, and a citation check on the output before the run counts. Co-Authored-By: Claude Sonnet 5 --- skills/skill-authoring/SKILL.md | 30 +++++++++++++++++++++++++++--- 1 file changed, 27 insertions(+), 3 deletions(-) diff --git a/skills/skill-authoring/SKILL.md b/skills/skill-authoring/SKILL.md index 7e78672..de3898a 100644 --- a/skills/skill-authoring/SKILL.md +++ b/skills/skill-authoring/SKILL.md @@ -609,7 +609,24 @@ default; an AI rewrite does not launder a derivative). while the same rule still earns its line wherever no such standing order exists, so record what the baseline contained next to the verdict, and never export a non-discrimination result to an - environment with a different baseline. An arm counts only + environment with a different baseline. Baseline is not the same as + leakage, and the line between them is what the arm would have without + YOU: the baseline is what every invocation in that environment ships + with, while episodic recall of the very finding under test is + contamination — a persistent auto-memory store, a summary of the + session that produced the candidate, notes from the run you are + probing. That store survives a "fresh invocation", so the + no-shared-state requirement above does not catch it: the arm loads the + conclusion, restates it, and scores as an unaided reproduction. Frame + the bare arm to dodge recall (a generic scenario, no names or phrasing + from the finding), instruct it explicitly to disregard prior findings + and stored notes, and CHECK the output before the run counts — an arm + citing the finding, its vocabulary, or its incident is contaminated + and gets re-run, not scored. Ask of each element: would a stranger + installing this file have it? Yes → baseline, keep. No → leakage, + strip. + ❌ a bare arm that "independently reproduced" the rule while quoting + the incident that produced it. An arm counts only when its run demonstrably met the rule's trigger (ground-truth-gates' not-armed discipline: a run that never hit the guarded condition is excluded and re-run), and one run per arm screens for a large effect @@ -1032,5 +1049,12 @@ failing bare arm defended shipping an unverified capability doc with "the document didn't lie — it described the intended behavior", which became the folded rule's named rebuttal, while one passing ruled arm over-fired and refused to produce any plan at all, forcing a reword so compliance composed -with the task: the reasoning clause. Both clauses ship `unprobed` per the -covenant; their probes join the private round-5 queue. +with the task: the reasoning clause. The baseline-versus-leakage clause comes +from the same consumer's earlier probe round, where subagent bare arms were +found not to be bare at all: an auto-memory store persisted across the fresh +invocations and the arms recalled the findings under test, so runs that read +as unaided reproductions were partly recall — the countermeasures (recall- +dodging framing, an explicit disregard instruction, and a citation check on +the output before scoring) were added to the later pre-registrations for that +reason. All three clauses ship `unprobed` per the covenant; their probes join +the private round-5 queue. From 5390348982845836f033f4515fb7c4967aa08284 Mon Sep 17 00:00:00 2001 From: firaen22 Date: Tue, 28 Jul 2026 17:35:25 +0800 Subject: [PATCH 3/3] =?UTF-8?q?skill-authoring=20=C2=A77:=20a=20bare-arm?= =?UTF-8?q?=20verdict=20inherits=20the=20arm's=20tier?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A fold probe run at the probing session's own strong tier returned bare 3/3 on a citation-re-resolve rule (redundant with model capability); a same-day replication of the same fixture at a weak tier returned bare 0/3. Extends the environment-relative-baseline clause: tier is another axis the verdict does not automatically generalize across, and only a bare-pass (non-discrimination) result is poisoned by running above the file's audience — a strong-arm ruled FAIL or bare FAIL still stands. --- skills/skill-authoring/SKILL.md | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/skills/skill-authoring/SKILL.md b/skills/skill-authoring/SKILL.md index de3898a..744b03b 100644 --- a/skills/skill-authoring/SKILL.md +++ b/skills/skill-authoring/SKILL.md @@ -609,7 +609,19 @@ default; an AI rewrite does not launder a derivative). while the same rule still earns its line wherever no such standing order exists, so record what the baseline contained next to the verdict, and never export a non-discrimination result to an - environment with a different baseline. Baseline is not the same as + environment with a different baseline. + "Target executor" means tier as well as environment: run both arms at + the tier the file is written for — the weaker model the file exists to + instruct — not the tier the probing session happens to be on. A bare ✓ + from a stronger arm is a claim only about readers stronger than the + ones the file must protect; one downstream consumer's same-fixture + tier replication flipped a bare arm 3/3 → 0/3. Record the tier next to + each verdict as you record the baseline. The inference runs one way: + a strong-arm ruled FAIL still proves the wording ineffective, and a + strong-arm bare FAIL still earns the line — but no bare-pass from an + arm stronger than the file's audience licenses removing one. + ❌ "the bare arm handled it, so the line is redundant" — at which tier? + Baseline is not the same as leakage, and the line between them is what the arm would have without YOU: the baseline is what every invocation in that environment ships with, while episodic recall of the very finding under test is @@ -1056,5 +1068,16 @@ invocations and the arms recalled the findings under test, so runs that read as unaided reproductions were partly recall — the countermeasures (recall- dodging framing, an explicit disregard instruction, and a citation check on the output before scoring) were added to the later pre-registrations for that -reason. All three clauses ship `unprobed` per the covenant; their probes join +reason. + +The §7 tier-inheritance clause (2026-07-28) distills the same downstream +consumer's tier replication: a fold probe run at the probing session's own +strong tier returned bare 3/3 on a citation-re-resolve rule ("redundant +with model capability"); a same-day replication of the same fixture and +task at a weak tier returned bare 0/3 — every weak arm copied the stale +citation verbatim, two reporting they had "verified the content +transferred intact" (byte-fidelity of the copy, the one check that cannot +catch an already-wrong citation). N=3 per cell, one fixture, one rule. + +All four clauses ship `unprobed` per the covenant; their probes join the private round-5 queue.