Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 45 additions & 5 deletions docs/sleep/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -308,7 +308,9 @@ This runs one additional consolidation per group (including a catch-all group wh
hinted and unhinted evidence are mixed), so it multiplies backend calls and token
use; configured dream rollouts and synthetic variants multiply the per-group work
too. Each group inherits the configured edit budget, gate mode/metric,
`gate_no_regression`, `dream_rollouts`, `dream_factor`, `recall_k`, and
`gate_no_regression`, `dream_rollouts`, `dream_factor`, `dream_adversarial`,
`dream_adversarial_blocking`, `dream_adversarial_margin`,
`dream_adversarial_rollouts`, `recall_k`, and
`evolve_skill`. Recalled archive tasks are restricted to that same skill hint;
shared memory is read-only in fan-out runs. Setting `evolve_skill` to `false`
therefore disables per-skill proposals as well as the managed skill proposal.
Expand All @@ -327,17 +329,55 @@ Resolution searches existing project-native `.agents/skills`, `.claude/skills`,
home and plugin-cache roots. Add repeatable `--skill-root PATH` values when an
integration stores skills elsewhere. Relative roots resolve below `--project`.

### Opt-in: experience replay & dream rollouts
### Opt-in: experience replay, dream rollouts, and robustness probes

Two consolidation mechanisms, both default **off** (behavior is unchanged unless you
enable them). They strengthen the nightly update when your tasks have a clean
correctness signal; the validation gate still governs what ships.
These controls are default **off** (behavior is unchanged unless you enable
them). Replay and rollouts strengthen the training signal; adversarial probes
measure candidate robustness. The validation gate still governs what ships
unless explicit adversarial blocking adds a second rejection condition.

| Config knob | Default | Effect |
|---|---|---|
| `dream_rollouts` | `1` | Run each task K times → learn from the good-vs-bad contrast (contrastive reflection). |
| `recall_k` | `0` | Associative recall — pull the K most-similar past tasks (from a persisted archive) into tonight's dream. |
| `dream_factor` | `0` | Add N lightweight synthetic variants of each task. |
| `dream_adversarial` | `0` | Score up to N harmless request-frame variants per real training task against each gate-eligible candidate. The factor is capped at 3 per task and 256 probes per candidate. |
| `dream_adversarial_blocking` | `false` | When `true`, reject a candidate whose brittleness is candidate-introduced under the baseline-relative rule below. When `false`, surface the same evidence without changing the gate decision. Requires `dream_adversarial_rollouts >= 2`. |
| `dream_adversarial_margin` | `0.0` | Tolerated worsening of the candidate gap relative to the baseline gap, in `[0, 1]`, before a row is marked brittle. Calibrate it on your own task mix before enabling blocking. |
| `dream_adversarial_rollouts` | `1` | Repeated samples per task and arm (capped at 8). Blocking requires at least 2 so one stochastic sample can never reject a candidate. |

Adversarial probes preserve the source reference and judge but change only the
request frame (for example, removing explicitly politeness-marked boilerplate
or adding request delimiters). They are generated from real, underived
training tasks only. Recalled, already-synthetic, validation, and test tasks
are excluded.

The decision is **baseline-relative** so pre-existing frame sensitivity never
flags a candidate: every source/probe pair is scored under both the current
(baseline) documents and the candidate documents, each score is the mean of
`dream_adversarial_rollouts` repeated samples, and a row is brittle only when
the candidate's probe-minus-source gap worsens beyond the margin relative to
the baseline gap AND the worsening holds in a strict majority of rollout
indices. All four aggregated scores and the per-rollout samples are retained
in the evidence so the decision is auditable. Any non-finite score fails
closed.

Probes are advisory first because any fixed robustness suite is an incomplete
proxy; enable blocking only after reviewing the advisory evidence and
calibrating the margin on your task mix. Blocking mode fails closed if no
eligible probe can be generated. The replay cost per gate-eligible candidate
is `rollouts * 2 * (sources + probes)`, so token and latency cost grow with
the number of real training tasks, the factor, and the rollout count.

Example `~/.skillopt-sleep/config.json`:

```json
{
"dream_adversarial": 2,
"dream_adversarial_blocking": false,
"dream_adversarial_margin": 0.015
}
```

### Paired A/B evalkit

Expand Down
4 changes: 3 additions & 1 deletion docs/sleep/multi-skill-staging.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,9 @@ with a note in both report formats. They never fall back to the managed skill's
document. The managed catch-all remains on `proposed_SKILL.md` and is not
duplicated as a per-skill row.

Each group inherits `recall_k`, `dream_rollouts`, `dream_factor`, `edit_budget`,
Each group inherits `recall_k`, `dream_rollouts`, `dream_factor`,
`dream_adversarial`, `dream_adversarial_blocking`,
`dream_adversarial_margin`, `dream_adversarial_rollouts`, `edit_budget`,
`gate_mode`, `gate_metric`, `gate_mixed_weight`, `gate_no_regression`, and
`evolve_skill`. Recalled archive tasks are restricted to the same skill hint,
and shared memory is read-only during group runs. Consequently,
Expand Down
Loading