diff --git a/CHANGELOG.md b/CHANGELOG.md index 066bdf3..80a3fe6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,21 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ## [Unreleased] +### Fixed + +- `SessionStart` hook output now reaches the model as `hookSpecificOutput.additionalContext` instead + of the terminal-only `systemMessage` chip, so the pinned-conventions index and meta-instruction + actually seed the conversation on session start. (#N) + +### Changed + +- `PostCompact` no longer emits a hook payload. Claude Code's hook output validator rejects + `hookSpecificOutput` for this event, leaving only the chip-only `systemMessage` envelope — which + never reaches the model and renders as long terminal noise on every `/compact`. The handler now + does its dedup-file truncation silently; on-demand pattern injection via PreToolUse continues to + work. The always-on pinned tier is unavailable post-compact pending a future re-prime mechanism. + See `docs/hook-pipeline-reference.md` and `ROADMAP.md`. (#N) + ## [0.4.0] - 2026-05-19 ### Changed diff --git a/ROADMAP.md b/ROADMAP.md index 8b96961..ca95f63 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -4,6 +4,16 @@ ## Future +- [ ] PostCompact re-prime workaround — Claude Code's hook output validator rejects + `hookSpecificOutput` for the PostCompact event, leaving `systemMessage` as the only envelope + and the pinned-conventions tier unable to re-seed the model context after `/compact`. Options + to investigate: (a) wait for Claude Code to accept `additionalContext` for PostCompact; (b) + re-prime opportunistically on the first PreToolUse after compaction by detecting a truncated + dedup file and injecting the pinned tier as additionalContext alongside the normal search + results; (c) a `lore reprime`-style agent-callable surface the model can invoke when it + notices its context has been compacted. Surfaced in PR #N (SessionStart envelope fix); see + `docs/hook-pipeline-reference.md` for the limitation as it stands today. + - [ ] Pre-release UX polish (deferred from edge-case-handling brainstorm) — friendlier empty-directory copy beyond the current tier-2 warning, empty-DB search hints, and a structured `SlugCollisionError` type (with `existing_path` / `existing_title` fields for diff --git a/docs/hook-pipeline-reference.md b/docs/hook-pipeline-reference.md index 55d677e..783b107 100644 --- a/docs/hook-pipeline-reference.md +++ b/docs/hook-pipeline-reference.md @@ -17,15 +17,34 @@ event payload from stdin and writes structured JSON to stdout. | Event | Output field | Matcher | Purpose | | ------------ | ------------------- | ------------------- | ---------------------------------------------------------------------- | -| SessionStart | `systemMessage` | All tools | Primes the session with a pattern index and meta-instruction | +| SessionStart | `additionalContext` | All tools | Primes the session with a pattern index and meta-instruction | | PreToolUse | `additionalContext` | `Edit\|Write\|Bash` | Searches for relevant patterns and injects them before the tool runs | | PostToolUse | `additionalContext` | `Bash` | Searches for patterns related to Bash errors (non-zero exit code only) | -| PostCompact | `systemMessage` | All tools | Re-primes the session after context compression | +| PostCompact | _(none)_ | All tools | Resets the per-session dedup file; emits no hook output (see below) | + +> **Why `additionalContext` for the first three events?** Claude Code routes +> `hookSpecificOutput.additionalContext` into the model's conversation as a system reminder, while +> the alternate `systemMessage` envelope renders as a transient terminal chip and never reaches the +> model. SessionStart, PreToolUse, and PostToolUse all use `additionalContext` so their payloads +> seed or augment the agent's context. Earlier releases mistakenly used `systemMessage` for +> SessionStart, so the pinned-conventions index never entered the conversation; this was corrected +> in 0.4.1. +> +> **Why PostCompact emits nothing.** Claude Code's hook output validator rejects +> `hookSpecificOutput` for the PostCompact event, leaving only the chip-only `systemMessage` +> envelope — which never reaches the model and would render as a long, recurring terminal chip with +> no use to either user or agent. Rather than emit noise, the handler suppresses output entirely and +> limits itself to its load-bearing side effect: truncating the per-session dedup file so the next +> PreToolUse re-injects patterns the agent saw before compaction. The always-on pinned tier is +> therefore unavailable post-compact, but on-demand injection continues to work. The hook is +> retained as an extension point: when Claude Code accepts `additionalContext` for PostCompact, or +> when a cleverer re-prime mechanism becomes available, the handler is the right place to plug it +> in. The roadmap tracks candidate workarounds. ### SessionStart Fires once at the beginning of every session. The hook creates (or truncates) the session -deduplication file, then returns a `systemMessage` containing: +deduplication file, then returns an `additionalContext` payload containing: - A meta-instruction telling the agent that patterns are injected automatically and should be followed as default conventions @@ -96,9 +115,20 @@ PostToolUse does not fire for successful commands or for non-Bash tools. ### PostCompact Fires when the agent's context window is compressed (a natural event during long sessions). The hook -truncates the deduplication file and re-emits the same content as SessionStart — the full pattern -index and meta-instruction. This ensures the agent retains awareness of the knowledge base even -after earlier injections have been compressed away. +truncates the per-session deduplication file so subsequent PreToolUse calls re-inject patterns the +agent saw before compaction. + +The handler produces no hook output. Claude Code's validator rejects `hookSpecificOutput` for +PostCompact, and the alternate `systemMessage` envelope would render as a long terminal chip that +never reaches the model — pure noise with no audience. Suppressing the output keeps the terminal +clean and is honest about the harness limitation: the always-on pinned-conventions tier cannot be +re-seeded after compaction with the channels currently available. On-demand injection via PreToolUse +continues to work normally. + +The handler is retained as an extension point — when Claude Code accepts `additionalContext` for +PostCompact, or when a different re-prime mechanism becomes feasible (a `lore reprime` +agent-callable surface, an opportunistic re-seed on the first PreToolUse after a truncated dedup +file is observed, or similar), this is where it plugs in. See the corresponding roadmap entry. ## Engine and Adapter diff --git a/docs/plans/2026-05-22-001-fix-sessionstart-additionalcontext-envelope-plan.md b/docs/plans/2026-05-22-001-fix-sessionstart-additionalcontext-envelope-plan.md new file mode 100644 index 0000000..3840ffc --- /dev/null +++ b/docs/plans/2026-05-22-001-fix-sessionstart-additionalcontext-envelope-plan.md @@ -0,0 +1,285 @@ +--- +title: "fix: Switch SessionStart and PostCompact to additionalContext envelope" +type: fix +status: completed +created: 2026-05-22 +completed: 2026-05-22 +--- + +# fix: Switch SessionStart and PostCompact to additionalContext envelope + +## Problem Frame + +Lore's `SessionStart` hook (and `PostCompact`) emits its payload via `{"systemMessage": "..."}`. In +Claude Code's hook protocol, `systemMessage` is a terminal-only channel: it renders as a transient +chip to the user but never enters the model's conversation context. The lore intro and the +available-patterns index have therefore been invisible to Claude in every workspace, on every +`SessionStart` source (`startup`, `resume`, `clear`, `compact`) and on `PostCompact`, for the +lifetime of the integration. + +Working envelope for context injection is +`{"hookSpecificOutput": {"hookEventName": "", "additionalContext": "..."}}` — already used +correctly by lore's `PreToolUse` / `PostToolUse` paths and by peer plugins (e.g. superpowers) for +`SessionStart`. + +Full diagnosis with transcript evidence: `tmp/lore-sessionstart-context-bug.md`. + +## Summary + +Change the `SessionStart` and `PostCompact` handlers in `src/hook.rs` to emit +`HookSpecific { additional_context, hook_event_name }` instead of `SystemMessage`. Delete the +now-unused `HookOutput::SystemMessage` variant and collapse `HookOutput` accordingly — no caller in +the codebase uses the chip-only channel after this change, and nothing in the configured plugin set +emits to it either. + +`format_session_context` (the function that builds the payload string) is unchanged — only the +output envelope changes. + +## Scope + +**In scope** + +- `src/hook.rs` — `handle_session_start` and `handle_post_compact` switch to the `HookSpecific` + envelope. +- `src/hook.rs` — remove `HookOutput::SystemMessage` variant; collapse the `untagged` enum to a + single struct shape (or single-variant enum) that serialises to `{"hookSpecificOutput": …}`. +- `tests/hook.rs` — update every assertion that reads `parsed["systemMessage"]` to read + `parsed["hookSpecificOutput"]["additionalContext"]`, and the matching `hookEventName` field where + useful as a regression guard against future event/envelope mismatches. +- Doc comments on `HookOutput` and the two handlers. +- `docs/hook-pipeline-reference.md` — the canonical event table and the `SessionStart` / + `PostCompact` sections currently document `systemMessage` as the output field. Update to + `additionalContext` and add a short note that the payload now lands in the model's context as a + system reminder rather than a terminal chip. +- `CHANGELOG.md` — single user-facing entry under `[Unreleased]`. +- `ROADMAP.md` — no change needed (this is a bug fix, not a roadmap item). + +**Out of scope** + +- `format_session_context` behaviour, content, or truncation logic. +- `PreToolUse` / `PostToolUse` handlers — already correct. +- `hooks.json` config — no source matcher change needed; the harness delivers the same set of events + regardless of envelope. +- The `tmp/lore-sessionstart-context-bug.md` bug doc — leave as-is for posterity; no need to move + it. + +### Deferred to Follow-Up Work + +None. The fix is self-contained. + +## Release posture + +This fix is release-worthy on its own. The `SessionStart` / `PostCompact` pipeline has shipped +non-functional for the entire life of the integration — every prior release silently dropped the +pinned-conventions index and the meta-instruction on the floor. The behavioural delta on landing is +large enough (universals now actually seed the model context; agent adherence to pinned conventions +becomes observable from turn one) that bundling it into the next opportunistic release risks burying +it under unrelated work in the release notes. + +After U1 + U2 land and the PR merges, cut a patch release per `docs/release-process.md`. Version +bump is a patch (`0.4.0` → `0.4.1`): the public CLI surface and configuration shape are unchanged; +only hook-output JSON shifts, and that channel is the harness's contract with Claude Code rather +than a documented user-facing API. + +The CHANGELOG entry written in U2 carries the release notes content; no separate release-notes pass +is required. + +## Key Technical Decisions + +**Delete `HookOutput::SystemMessage`, do not retain as escape hatch.** Why: zero callers after this +change, and no other plugin in the configured set emits a chip via this envelope either. Keeping the +variant as "documented future flexibility" is speculative — re-adding it later when a real chip-only +event appears is a five-line struct, and a real caller will teach us the correct shape (per-event +matcher, level parameter, etc.) better than a preserved-just-in-case variant. The bug doc's +suggestion to retain it was a conservative aside, not load-bearing analysis. (Origin discussion: +scoping confirmation, this session.) + +**Collapse the `untagged` enum if only one shape remains.** With `SystemMessage` removed, +`HookOutput` becomes a single-variant enum. Replace with a plain struct wrapping +`HookSpecificOutput`, or keep as a single-variant enum if call sites read more cleanly that way. The +implementer picks based on what reads best at the four `Ok(Some(...))` return sites in +`src/hook.rs`. + +**`hookEventName` per call site.** The inner `hookEventName` must match the event being handled: +`"SessionStart"` in `handle_session_start`, `"PostCompact"` in `handle_post_compact`. This is +already how the `PreToolUse` / `PostToolUse` paths set it. + +## Implementation Units + +```mermaid +graph LR + U1[U1. Code fix] --> U2[U2. Docs and CHANGELOG] +``` + +U2 follows U1 so the documentation describes the shipped behaviour, not a hypothetical state. + +### U1. Switch hook envelopes and remove SystemMessage variant + +**Goal:** `SessionStart` and `PostCompact` deliver their content as `additionalContext` so the model +sees it. Dead chip-only variant removed. + +**Files:** + +- `src/hook.rs` (modify) +- `tests/hook.rs` (modify — assertion shape only) + +**Approach:** + +- Replace the `Ok(Some(HookOutput::SystemMessage { … }))` at `src/hook.rs:161` with the + `HookSpecific` envelope, `hookEventName = + "SessionStart"`, `additionalContext = context`. +- Same change at `src/hook.rs:480` for `PostCompact`, with `hookEventName = "PostCompact"`. +- Remove the `SystemMessage` variant from the `HookOutput` enum (`src/hook.rs:62-65`) and its doc + comment lines (`src/hook.rs:53-54`). If `HookOutput` now has a single variant, collapse to a + struct or keep as a one-variant enum — implementer's call based on which reads more cleanly at the + four return sites. +- Update the doc comment on `HookOutput` to reflect that all events use the `hookSpecificOutput` + envelope. +- In `tests/hook.rs`, replace every read of `parsed["systemMessage"]` with + `parsed["hookSpecificOutput"]["additionalContext"]`. Touch points identified by + `git grep -nF systemMessage tests/`: lines 245, 247, 283, 327, 361, 363, 668, 669, 793, 797, 798, + 998, 1059, 1287, 1568. The line at 1568 is an assertion-message string only — update for + consistency, no behavioural impact. +- For `SessionStart`-emitting tests and the `PostCompact` test, also assert that + `parsed["hookSpecificOutput"]["hookEventName"]` equals the expected event name. This guards + against future copy-paste mistakes that would put `"SessionStart"` on a `PostCompact` payload + (which the harness silently ignores, reproducing the same class of bug). + +**Test scenarios:** + +- `SessionStart` JSON output has `hookSpecificOutput.additionalContext` containing the lore intro + substring; no top-level `systemMessage` key present. +- `SessionStart` JSON output has `hookSpecificOutput.hookEventName == "SessionStart"`. +- `PostCompact` JSON output has `hookSpecificOutput.additionalContext` matching the `SessionStart` + content for the same DB state (existing parity test at line ~793 continues to hold, just reading + the new shape). +- `PostCompact` JSON output has `hookSpecificOutput.hookEventName == "PostCompact"`. +- Render-cap truncation test (line ~1565) still triggers and the truncation marker still appears in + the new `additionalContext` field. +- All existing `PreToolUse` / `PostToolUse` tests continue to pass unchanged — they already use + `hookSpecificOutput` and the enum collapse must not regress them. + +**Verification:** + +- `cargo test --test hook` green, with the renamed assertions in place. +- `cargo build` clean, no unused-variant or dead-code warnings. +- Manual: in a fresh Claude Code session with this build of `lore` installed, run the acceptance + prompt from the bug doc verbatim: + > Before doing anything else — no tool calls, no MCP, no file reads — tell me verbatim: do you + > see, in your initial context, a system message that begins with the words "This project uses + > lore for the author's strong coding preferences"? Quote the first sentence back if yes; say "no" + > if not. Expected: yes, with the first sentence quoted. +- Repeat the acceptance prompt after `/clear` and `/compact`. Both should answer yes. +- Confirm the transient terminal chip is gone (intended trade-off — the payload is too long to be + useful as a chip). + +### U2. Update hook-pipeline reference and changelog + +**Goal:** Documentation matches the shipped envelope. Release notes record the behavioural change +for users who notice the terminal chip disappear and the pinned conventions begin to take hold. + +**Dependencies:** U1. + +**Files:** + +- `docs/hook-pipeline-reference.md` (modify) +- `CHANGELOG.md` (modify) + +**Approach:** + +- In `docs/hook-pipeline-reference.md`, replace the two `systemMessage` cells in the event table at + lines 20 and 23 with `additionalContext`. +- In the `SessionStart` section (line 25 onwards), reword the "returns a `systemMessage` + containing…" sentence to "returns an `additionalContext` payload containing…". The bulleted + content description below is unaffected. +- In the `PostCompact` section (line 96 onwards), the prose at line 99 already says "re-emits the + same content as SessionStart" — extend the same envelope clarification here so a reader sees both + events use the same field. +- Add one short blockquote aside near the event table noting that the `SessionStart` and + `PostCompact` payloads enter the model's context as a system reminder, not a transient terminal + chip. This is the load-bearing user-visible change and merits an explicit callout given how long + the previous behaviour shipped. +- Sweep the rest of the file with `git grep -nF systemMessage + docs/` to confirm no stale + references remain. +- In `CHANGELOG.md`, add an entry under `[Unreleased]` in the `Fixed` group following the project + convention (assertive voice, one sentence, ends in `(#N)` once the PR is opened). Suggested + wording: + > `SessionStart` and `PostCompact` hook output now lands in the model's conversation context as + > `additionalContext` instead of a terminal-only `systemMessage` chip, so the pinned-conventions + > index and meta-instruction actually reach the agent on session start and after compaction. (#N) + +**Test scenarios:** + +- `Test expectation: none -- documentation and changelog edits with no + executable surface.` + Verification is the post-fix grep and a human read pass. + +**Verification:** + +- `git grep -nF systemMessage docs/` returns zero hits. +- `git grep -nF systemMessage README.md ROADMAP.md CONTRIBUTING.md` returns zero hits (confirms the + surface really was confined to the pipeline reference; if any other surface turns up, fold it into + this unit before commit). +- `CHANGELOG.md` `[Unreleased]` section contains the new `Fixed` bullet, and the bullet renders + correctly under Keep a Changelog conventions (one assertive sentence, PR number suffix). +- `mdbook build docs` (or whatever the project uses to render the documentation tree) is clean — no + broken anchor references introduced by the reword. + +## Risks + +- **Test churn surface.** ~15 assertion sites in `tests/hook.rs` reference `systemMessage`. A + `git grep` after the edit pass must return zero hits before commit. +- **Envelope mismatch on `PostCompact`.** Easy to leave `hookEventName: "SessionStart"` in the + `PostCompact` handler from a copy-paste. The harness will silently ignore mismatched event names, + reproducing the exact class of bug this plan fixes. The per-event `hookEventName` assertion in + U1's test scenarios catches this. +- **Plugin trace records.** Existing trace records under `$XDG_STATE_HOME/lore/traces/` were written + with the old envelope. No migration is needed — traces are session-scoped and self-healing on new + sessions. Mentioned for awareness, not action. + +## Post-implementation notes + +The plan body above is preserved as the planning-time decision artefact. The shipped state diverges +from it on one axis: live testing of `/compact` surfaced that Claude Code's hook output validator +**rejects** `hookSpecificOutput` for the `PostCompact` event (while accepting it for `SessionStart`, +`PreToolUse`, `PostToolUse`, and others). There is no envelope that delivers `PostCompact` output to +the model. + +Shipped resolution: + +- `SessionStart` switched to `hookSpecificOutput.additionalContext` as planned. The original bug is + fixed: the lore intro and pinned-conventions index now reach the model on every `SessionStart` + source. +- `PostCompact` produces no hook output at all. The dedup-file truncation still runs as a side + effect, so the next `PreToolUse` re-injects relevant patterns; the always-on pinned tier is + unavailable post-compact pending a future re-prime mechanism. The handler is retained as an + extension point. +- The "delete `HookOutput::SystemMessage` variant" key decision from the plan still applies and was + honoured: with `PostCompact` returning `Ok(None)`, the variant has no callers and was removed. The + enum collapsed to a single-shape struct with one constructor (`additional_context`). +- A typed `HookEventName` enum (replacing the `&str` parameter) was deferred to the next PR that + adds a hook event, per code review. + +Documentation of the harness limitation: + +- `docs/hook-pipeline-reference.md` event table marks `PostCompact`'s output field as _(none)_ with + an explanatory blockquote. +- `ROADMAP.md` carries a `PostCompact re-prime workaround` entry listing candidate approaches. +- `docs/solutions/integration-issues/verify-host-delivery-not-just-hook-exit-2026-05-22.md` captures + the broader learning about verifying host delivery rather than trusting upstream signals like + `exitCode=0` and visible terminal chips. +- The companion pattern at `agents/claude-code-hook-output-envelopes.md` in the lore-patterns + repository documents the per-event envelope acceptance map for future integrations. + +## References + +- Bug context: `tmp/lore-sessionstart-context-bug.md` +- Lore hook source: `src/hook.rs:57-75` (envelope types), `:161` (`SessionStart` return), `:480` + (`PostCompact` return) +- Working envelope precedent in same file: `src/hook.rs:340`, `:567` (`PreToolUse` / `PostToolUse` + return sites) +- Test surface: `tests/hook.rs` — see line list in U1. +- Documentation surface: `docs/hook-pipeline-reference.md:18-23` (event table), `:25-39` + (`SessionStart` section), `:96-101` (`PostCompact` section). +- Changelog: `CHANGELOG.md` `[Unreleased]` section. diff --git a/docs/solutions/integration-issues/verify-host-delivery-not-just-hook-exit-2026-05-22.md b/docs/solutions/integration-issues/verify-host-delivery-not-just-hook-exit-2026-05-22.md new file mode 100644 index 0000000..fae8dbf --- /dev/null +++ b/docs/solutions/integration-issues/verify-host-delivery-not-just-hook-exit-2026-05-22.md @@ -0,0 +1,118 @@ +--- +title: "Verify host delivery, not just hook exit — successful hook execution is not evidence the model received the payload" +date: 2026-05-22 +category: integration-issues +module: integrations +problem_type: integration_issue +component: tooling +symptoms: + - "Hook exits 0 and harness logs hook_success, but the model behaves as if the payload was never injected" + - "Terminal renders a chip from the hook output, masking the absence of model-side delivery" + - "The bug persists across every workspace and every session for the integration's lifetime" +root_cause: silent_channel_misroute +resolution_type: testing_discipline +severity: high +tags: + - claude-code + - sessionstart + - postcompact + - additional-context + - system-message + - hooks + - integration-testing + - silent-failure +--- + +# Verify host delivery, not just hook exit + +## Problem + +Lore's `SessionStart` hook returned `{"systemMessage": "..."}` since the integration shipped. Claude +Code's harness routes `systemMessage` only to the terminal UI as a transient chip — the model never +sees it. The hook process exited `0`, the harness logged `hook_success`, the terminal rendered the +chip (sometimes), and every signal upstream of the model said "working." The pinned-conventions +index and the meta-instruction were invisible to the agent in every workspace, on every +`SessionStart` source (`startup`, `resume`, `clear`, `compact`), for the lifetime of the +integration. + +The bug was found accidentally: the user noticed the chip rendering looked inconsistent across +`SessionStart` sources, which surfaced the channel question, which led to a model-side test that +asked the agent to quote the payload — and it couldn't. + +## Symptoms + +- `lore hook` process exits `0`, `stdoutLen` is non-zero in the transcript jsonl, `hook_success` is + logged at every event. +- Terminal renders a chip for the payload on at least some events (often the ones with a quieter + screen — `/clear`, `/compact`). +- The model exhibits no awareness of the pinned conventions across sessions — agents repeatedly + violate conventions that the user knows are configured. +- The transcript jsonl records `content: ""` on the hook attachment row even though the hook stdout + was non-empty. + +## What Didn't Work + +- Reading the harness's printed hook-output schema dump on validation failure. The dump is + incomplete — it omits the `hookSpecificOutput` variants that work in practice for `SessionStart`, + so matching the dump literally produces a wrong envelope for some events. +- Treating `exitCode=0` plus terminal chip rendering as evidence of model- side delivery. The chip + is a separate channel; its visibility tells you nothing about what the model received. +- Inspecting the hook's stdout for "correct-looking" JSON. Both `{"systemMessage": "..."}` and + `{"hookSpecificOutput": {...}}` are syntactically valid; only the latter (and only on the right + events) reaches the model. + +## Solution + +For every host-integration channel that targets a specific consumer (typically the model in agent +harnesses), the only authoritative test is asking the consumer "did you receive it?" before the +integration ships. + +For Claude Code hooks specifically, paste this in a fresh session before any tool call: + +``` +Before doing anything else — no tool calls, no MCP, no file reads — tell me +verbatim: do you see, in your initial context, a system message that begins +with the words ""? Quote the first sentence back +if yes; say "no" if not. +``` + +Repeat across every event source the hook fires on (`startup`, `resume`, `clear`, `compact`). The +acceptance prompt costs one session per event and gives an unambiguous model-side answer. + +For non-agent host integrations (webhooks, message queues, plugin APIs), the equivalent is a +synthetic consumer that records what it received and asserts the payload matches expectation. The +shape of the test changes per host; the discipline does not. + +## Why This Matters + +Host integrations have multiple plausible delivery channels, and "the host didn't reject my call" is +a weak proof of delivery when the channels look syntactically interchangeable. Claude Code's +`systemMessage` and `hookSpecificOutput.additionalContext` are both top-level JSON keys with similar +shapes; both produce a `hook_success` on the harness side; both render output that a user might see. +Only one reaches the model. The asymmetry is structural to the host and invisible to upstream +signals. + +The lifetime of this bug — the integration's entire history — is the cost of trusting upstream +signals. The fix took an hour; the discovery took months of agent behaviour that was misattributed +to model capability rather than missing context. + +## Prevention + +- For any hook, plugin, or host integration where the payload targets a downstream consumer: write a + model-side (or consumer-side) acceptance test before ship, and re-run it after any envelope, + schema, or routing change in the host. +- When the host publishes multiple output channels (chip vs context, log vs reply, etc.), document + explicitly which channel each event uses and why. A table in the integration doc beats per-event + scattered comments. +- Be suspicious of host-published validator schemas. Reproduce the accepted shape empirically when + the documented schema looks incomplete or self-contradictory. +- Distrust "everything looks fine" when the consumer's behaviour is the only thing that can actually + confirm delivery. + +## Related + +- `agents/claude-code-hook-output-envelopes.md` in the lore-patterns repository captures the + per-event envelope acceptance map for Claude Code hooks specifically, including the `PostCompact` + rejection that followed from this same investigation. +- `integration-issues/additional-context-timing-in-pretooluse-hooks-2026-04-02.md` documents a + different shape of "hook fires but effect is not what you expect" for the `PreToolUse` event. diff --git a/src/hook.rs b/src/hook.rs index b6654fb..29552db 100644 --- a/src/hook.rs +++ b/src/hook.rs @@ -47,25 +47,36 @@ pub struct HookInput { /// Written to stdout as JSON. /// -/// Two variants: -/// - `HookSpecific` — for events that support `hookSpecificOutput` -/// (`PreToolUse`, `PostToolUse`). -/// - `SystemMessage` — for events where Claude Code only accepts a top-level -/// `systemMessage` field (`SessionStart`, `PostCompact`). +/// Every emitting hook event uses the `hookSpecificOutput.additionalContext` +/// envelope so the payload lands in the model's conversation as a system +/// reminder. `PostCompact` is the one event that suppresses output entirely +/// (Claude Code's validator rejects `hookSpecificOutput` there and the +/// chip-only `systemMessage` envelope never reaches the model — see +/// `docs/hook-pipeline-reference.md`). #[derive(Debug, Serialize)] -#[serde(untagged)] -pub enum HookOutput { - HookSpecific { - #[serde(rename = "hookSpecificOutput")] - hook_specific_output: HookSpecificOutput, - }, - SystemMessage { - #[serde(rename = "systemMessage")] - system_message: String, - }, +pub struct HookOutput { + #[serde(rename = "hookSpecificOutput")] + pub hook_specific_output: HookSpecificOutput, } -/// The payload nested inside `HookOutput::HookSpecific`. +impl HookOutput { + /// Build the `additionalContext` envelope for the given event. + /// + // The event name is a `&str` for now because there are only four valid + // values and all call sites use string literals. If a fifth event or a + // re-prime workaround lands, tighten this to a typed `HookEventName` enum + // to remove the typo class at compile time. + pub fn additional_context(hook_event_name: &str, additional_context: String) -> Self { + Self { + hook_specific_output: HookSpecificOutput { + hook_event_name: hook_event_name.to_string(), + additional_context, + }, + } + } +} + +/// The payload nested inside `HookOutput`. #[derive(Debug, Serialize)] pub struct HookSpecificOutput { #[serde(rename = "hookEventName")] @@ -158,9 +169,10 @@ fn handle_session_start( } } - Ok(Some(HookOutput::SystemMessage { - system_message: context, - })) + Ok(Some(HookOutput::additional_context( + "SessionStart", + context, + ))) } /// Handle `PreToolUse`: extract query, search, predicate-filter, dedup-filter, @@ -337,12 +349,7 @@ fn handle_pre_tool_use( // 9. Format and emit. let context = format_imperative(&combined); - Ok(Some(HookOutput::HookSpecific { - hook_specific_output: HookSpecificOutput { - hook_event_name: "PreToolUse".to_string(), - additional_context: context, - }, - })) + Ok(Some(HookOutput::additional_context("PreToolUse", context))) } /// Apply the universal-pattern predicate filter to a list of expanded chunks. @@ -454,10 +461,22 @@ fn expand_to_siblings(db: &KnowledgeDB, seeds: &[SearchResult]) -> Vec anyhow::Result> { let start = std::time::Instant::now(); @@ -469,17 +488,13 @@ fn handle_post_compact( lore_debug!("PostCompact dedup reset error: {e}"); } - let context = format_session_context(db, &config.knowledge_dir)?; - if config.trace_enabled() && let Some(session_id) = input.session_id.as_deref() { emit_post_compact_trace(session_id, start); } - Ok(Some(HookOutput::SystemMessage { - system_message: context, - })) + Ok(None) } /// Handle `PostToolUse`: on Bash errors, search with stderr and return patterns. @@ -564,12 +579,7 @@ fn handle_post_tool_use( } let context = format_imperative(&results); - Ok(Some(HookOutput::HookSpecific { - hook_specific_output: HookSpecificOutput { - hook_event_name: "PostToolUse".to_string(), - additional_context: context, - }, - })) + Ok(Some(HookOutput::additional_context("PostToolUse", context))) } /// Apply the per-class relevance floor: universal chunks are filtered against diff --git a/tests/hook.rs b/tests/hook.rs index 739c009..f8a1f42 100644 --- a/tests/hook.rs +++ b/tests/hook.rs @@ -242,9 +242,14 @@ fn hook_session_start_returns_meta_instruction() { let parsed: serde_json::Value = serde_json::from_str(&stdout) .unwrap_or_else(|e| panic!("stdout is not valid JSON: {e}\nstdout: {stdout}")); - let ctx = parsed["systemMessage"] + assert_eq!( + parsed["hookSpecificOutput"]["hookEventName"].as_str(), + Some("SessionStart"), + "SessionStart payload must carry hookEventName=SessionStart so Claude Code routes it into model context" + ); + let ctx = parsed["hookSpecificOutput"]["additionalContext"] .as_str() - .expect("SessionStart should return a top-level systemMessage"); + .expect("SessionStart should return hookSpecificOutput.additionalContext"); assert!( ctx.contains("lore for the author"), "should contain meta-instruction: {ctx}" @@ -280,7 +285,9 @@ fn hook_session_start_advertises_git_advisory_for_non_git_dir() { let stdout = String::from_utf8(output.get_output().stdout.clone()).unwrap(); let parsed: serde_json::Value = serde_json::from_str(&stdout).unwrap(); - let ctx = parsed["systemMessage"].as_str().unwrap(); + let ctx = parsed["hookSpecificOutput"]["additionalContext"] + .as_str() + .unwrap(); assert!( ctx.contains("not a git repository"), @@ -324,7 +331,9 @@ fn hook_session_start_omits_git_advisory_for_git_dir() { let stdout = String::from_utf8(output.get_output().stdout.clone()).unwrap(); let parsed: serde_json::Value = serde_json::from_str(&stdout).unwrap(); - let ctx = parsed["systemMessage"].as_str().unwrap(); + let ctx = parsed["hookSpecificOutput"]["additionalContext"] + .as_str() + .unwrap(); assert!( !ctx.contains("not a git repository"), @@ -335,7 +344,13 @@ fn hook_session_start_omits_git_advisory_for_git_dir() { } #[test] -fn hook_post_compact_returns_session_context() { +fn hook_post_compact_produces_no_output() { + // Claude Code's hook output validator rejects `hookSpecificOutput` for + // `PostCompact`, leaving only the chip-only `systemMessage` envelope — + // which never reaches the model and would render as a long, useless + // terminal chip. The handler suppresses output entirely; the only + // load-bearing PostCompact behaviour is dedup truncation, pinned by + // `hook_post_compact_resets_dedup_file` below. let (_tmp, config_path) = setup_test_env(); let input = serde_json::json!({ @@ -352,22 +367,10 @@ fn hook_post_compact_returns_session_context() { let stdout = String::from_utf8(output.get_output().stdout.clone()).unwrap(); assert!( - !stdout.is_empty(), - "PostCompact should produce output like SessionStart" - ); - - let parsed: serde_json::Value = serde_json::from_str(&stdout) - .unwrap_or_else(|e| panic!("stdout is not valid JSON: {e}\nstdout: {stdout}")); - let ctx = parsed["systemMessage"] - .as_str() - .expect("PostCompact should return a top-level systemMessage"); - assert!( - ctx.contains("lore for the author"), - "should contain meta-instruction: {ctx}" - ); - assert!( - ctx.contains("Available patterns:"), - "should list available patterns: {ctx}" + stdout.is_empty(), + "PostCompact must not emit a hook payload (harness validator rejects \ + hookSpecificOutput and systemMessage is pure terminal noise); \ + got: {stdout}" ); } @@ -665,8 +668,8 @@ fn hook_full_lifecycle_session_dedup_compact_reinject() { let start_parsed: serde_json::Value = serde_json::from_str(&start_stdout).unwrap(); assert!( - start_parsed["systemMessage"].is_string(), - "SessionStart should return a systemMessage" + start_parsed["hookSpecificOutput"]["additionalContext"].is_string(), + "SessionStart should return hookSpecificOutput.additionalContext" ); // 2. PreToolUse — first call should inject patterns. @@ -711,7 +714,11 @@ fn hook_full_lifecycle_session_dedup_compact_reinject() { ); } - // 4. PostCompact — should reset dedup and return session context. + // 4. PostCompact — should reset dedup. The handler suppresses hook output + // entirely (Claude Code rejects hookSpecificOutput for PostCompact and + // a systemMessage chip provides no model-facing value), so stdout must + // be empty. The lifecycle-relevant observable is the dedup reset, pinned + // by step 5 below. let compact_input = serde_json::json!({ "hook_event_name": "PostCompact", "session_id": session_id @@ -726,8 +733,8 @@ fn hook_full_lifecycle_session_dedup_compact_reinject() { let compact_stdout = String::from_utf8(compact_output.get_output().stdout.clone()).unwrap(); assert!( - !compact_stdout.is_empty(), - "PostCompact should produce output" + compact_stdout.is_empty(), + "PostCompact must produce no hook output; got: {compact_stdout}" ); // 5. Same PreToolUse again — after PostCompact reset, should re-inject. @@ -753,59 +760,6 @@ fn hook_full_lifecycle_session_dedup_compact_reinject() { let _ = std::fs::remove_file(dedup_path); } -#[test] -fn hook_session_start_and_post_compact_return_same_content() { - let (_tmp, config_path) = setup_test_env(); - let session_id = format!("same-content-test-{}", std::process::id()); - - let start_input = serde_json::json!({ - "hook_event_name": "SessionStart", - "session_id": session_id - }); - - let start_output = Command::cargo_bin("lore") - .unwrap() - .args(["hook", "--config", config_path.to_str().unwrap()]) - .write_stdin(serde_json::to_string(&start_input).unwrap()) - .assert() - .success(); - - let start_stdout = String::from_utf8(start_output.get_output().stdout.clone()).unwrap(); - - let compact_input = serde_json::json!({ - "hook_event_name": "PostCompact", - "session_id": session_id - }); - - let compact_output = Command::cargo_bin("lore") - .unwrap() - .args(["hook", "--config", config_path.to_str().unwrap()]) - .write_stdin(serde_json::to_string(&compact_input).unwrap()) - .assert() - .success(); - - let compact_stdout = String::from_utf8(compact_output.get_output().stdout.clone()).unwrap(); - - // Both should produce output. - assert!(!start_stdout.is_empty()); - assert!(!compact_stdout.is_empty()); - - // Both use systemMessage — content should be identical. - let start_parsed: serde_json::Value = serde_json::from_str(&start_stdout).unwrap(); - let compact_parsed: serde_json::Value = serde_json::from_str(&compact_stdout).unwrap(); - - let start_ctx = start_parsed["systemMessage"].as_str().unwrap(); - let compact_ctx = compact_parsed["systemMessage"].as_str().unwrap(); - assert_eq!( - start_ctx, compact_ctx, - "SessionStart and PostCompact should return the same context content" - ); - - // Clean up. - let dedup_path = lore::hook::dedup_file_path(&session_id); - let _ = std::fs::remove_file(dedup_path); -} - // --------------------------------------------------------------------------- // .test.ts TypeScript testing pattern // --------------------------------------------------------------------------- @@ -995,7 +949,10 @@ fn invoke_session_start(config_path: &Path, session_id: &str) -> String { let stdout = String::from_utf8(output.get_output().stdout.clone()).unwrap(); let parsed: serde_json::Value = serde_json::from_str(&stdout).unwrap(); - parsed["systemMessage"].as_str().unwrap().to_string() + parsed["hookSpecificOutput"]["additionalContext"] + .as_str() + .unwrap() + .to_string() } #[test] @@ -1038,36 +995,6 @@ fn hook_session_start_emits_pinned_section_with_body_above_index_when_universal_ ); } -#[test] -fn hook_post_compact_re_emits_pinned_section() { - let (_tmp, config_path) = setup_with_universal_pattern(); - - let input = serde_json::json!({ - "hook_event_name": "PostCompact", - "session_id": "test-post-compact-pinned", - }); - - let output = Command::cargo_bin("lore") - .unwrap() - .args(["hook", "--config", config_path.to_str().unwrap()]) - .write_stdin(serde_json::to_string(&input).unwrap()) - .assert() - .success(); - - let stdout = String::from_utf8(output.get_output().stdout.clone()).unwrap(); - let parsed: serde_json::Value = serde_json::from_str(&stdout).unwrap(); - let ctx = parsed["systemMessage"].as_str().unwrap(); - - assert!( - ctx.contains("## Pinned conventions"), - "PostCompact should re-emit the pinned section: {ctx}" - ); - assert!( - ctx.contains("Always push with `git push origin HEAD`"), - "pinned body should re-appear at PostCompact: {ctx}" - ); -} - // --------------------------------------------------------------------------- // Track 1B: predicated universals defer from SessionStart pinning to the // PreToolUse predicate path. universal_patterns() filters @@ -1262,50 +1189,15 @@ fn hook_session_start_skip_then_pre_tool_use_fire_couples_predicate_path() { let _ = std::fs::remove_file(&dedup_path); } -#[test] -fn hook_post_compact_excludes_predicated_universal_from_pinned_section() { - // R4 / hazard pin: PostCompact shares format_session_context with - // SessionStart today, but pin the invariant directly so a future refactor - // splitting the shared path cannot silently regress predicated-chunk - // filtering on the PostCompact side (composition-cascade mitigation). - let (_tmp, config_path) = setup_with_predicated_and_unpredicated_universals(); - - let input = serde_json::json!({ - "hook_event_name": "PostCompact", - "session_id": "test-track-1b-post-compact", - }); - - let output = Command::cargo_bin("lore") - .unwrap() - .args(["hook", "--config", config_path.to_str().unwrap()]) - .write_stdin(serde_json::to_string(&input).unwrap()) - .assert() - .success(); - - let stdout = String::from_utf8(output.get_output().stdout.clone()).unwrap(); - let parsed: serde_json::Value = serde_json::from_str(&stdout).unwrap(); - let ctx = parsed["systemMessage"].as_str().unwrap(); - - assert!( - ctx.contains("genuinely-universal-marker"), - "PostCompact should still re-emit the un-predicated universal body: {ctx}" - ); - assert!( - !ctx.contains("predicated-git-marker"), - "PostCompact must NOT re-emit the predicated universal body: {ctx}" - ); -} - #[test] fn hook_post_compact_resets_dedup_file() { - // Sibling to the predicated-universal hazard pin above. The R4 test - // covers the rendered-context filter; this one pins the orthogonal - // contract that PostCompact also truncates the per-session dedup file - // (so post-compaction tool calls re-inject patterns the agent saw before - // the compaction event). Without this assertion, a refactor that drops - // `reset_dedup` from `handle_post_compact` would not be caught by any - // of the existing PostCompact tests — they only inspect the system - // message payload. + // PostCompact's load-bearing contract. The handler produces no hook + // output (covered by `hook_post_compact_produces_no_output`), so this is + // the only externally observable behaviour that matters: the per-session + // dedup file must be truncated so the next PreToolUse re-injects + // patterns the agent saw before compaction. A refactor that drops + // `reset_dedup` from `handle_post_compact` would otherwise pass every + // remaining PostCompact test in this file. let (_tmp, config_path) = setup_with_predicated_and_unpredicated_universals(); let session_id = format!("test-track-1b-post-compact-truncate-{}", std::process::id()); let dedup_path = lore::hook::dedup_file_path(&session_id); @@ -1565,7 +1457,7 @@ fn hook_session_start_truncates_pinned_section_at_render_budget() { assert!( ctx.contains("_[pinned conventions truncated at 32768 bytes"), "expected truncation marker once cumulative body crossed 32 KB; \ - got {} bytes of systemMessage starting {:?}", + got {} bytes of additionalContext starting {:?}", ctx.len(), &ctx.chars().take(200).collect::(), ); @@ -1794,7 +1686,7 @@ fn hook_pre_tool_use_universal_persists_after_post_compact_truncation() { .success(); } - // PostCompact (truncates dedup, re-emits SessionStart content) + // PostCompact (truncates dedup; emits no hook output) let post_compact = serde_json::json!({ "hook_event_name": "PostCompact", "session_id": session_id,