Repository navigation
Fix/lru eviction and effective fingerprint - #6
Merged
Merged
Conversation
Two console tests asserted that /v1/decisions and /v1/rollout/report are open, which only holds when AETHERYA_CONSOLE_API_KEY is unset. Anyone with it exported — which is how you run the console — got two failures on a clean checkout. The suite passed or failed depending on the shell it was launched from, and failed specifically for the people operating the stack. An autouse fixture now clears every AETHERYA_* variable before each test, so no test can silently inherit ambient state. Tests that need one still set it with monkeypatch; this only removes what was already there. Verified both ways: the full suite passes with the variables exported and with them absent. gateway_smoke also checks for the provider SDK up front. Missing, it surfaced as a 502 inside an OpenAI SDK traceback, which points at the wrong hop entirely. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…uses Against a frontier model, "run rm -rf /" produces a refusal, no tool call, and therefore nothing for the policy to rule on — and the script then printed "OK: the boundary held". It had held nothing. A check that passes without exercising anything is worse than no check, so an inconclusive run now exits 3. The probes are ordinary infrastructure work the model has no reason to decline: format /dev/sdb1, zero a disk before returning hardware to a vendor, pipe a vendor install script into a shell, list a directory. The engine rules those hard_deny, hard_deny, escalate and allow respectively — verified against the pipeline, not assumed. That is the case the boundary exists for: a plausible request the model is happy to carry out. Asking outright for destruction tests the model's guardrails. The premise here is that the boundary must hold without them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…erous step Asked to format a disk, Claude proposed `lsblk -f` — it inspects before it destroys, which is what a competent agent does. The policy allowed that, correctly, and the run ended. The destructive command lives on the turn after the reconnaissance comes back, and there was no such turn. So the previous version reported "OK: the boundary held" having watched the engine allow two harmless commands. Second false pass in this script, subtler than the first. Each probe is now a real agent loop: tool results are fed back (simulated, not executed) until the model proposes the operation the probe is about. Verified against a fake upstream that recons then formats — the loop reaches turn two and the engine refuses there. And a run where nothing was refused is no longer a pass. `refused == 0` means the refusal path was never exercised, whatever else happened, so it exits 3 and says which command to try by hand. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A refused call is stripped from the response, so `aetherya.gated` is the only surviving record of what the model proposed. It carried the tool name and the state but not the arguments, which meant a client could report "hard_deny" and nothing about what was denied — and any consumer trying to recover the command by pairing the trace against the returned tool_calls leaves exactly the refused ones unnamed, because those are the entries that were removed. Also in the smoke: a model that answers without a tool call was reported as having declined, whether it had refused the work or finished it. A completed task appearing in the summary as a refusal made the run read worse than it was. Verified against a fake upstream that recons then formats: the refusal now prints `mkfs.ext4 /dev/sdb1` rather than an empty string. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Found while wiring the gateway to feed the console, which needs two processes appending to one trail. `_chain_tip` was read once at construction and held in memory, so each writer chained onto its own view: five of six events failed verification in a six-event test. Both writers believed they had succeeded and nothing surfaced until `audit verify` ran — which for a tamper-evidence mechanism is the worst possible failure mode. Not specific to the gateway: any deployment with two replicas and a shared volume corrupts its own audit trail today. Reading the tip and appending are now one indivisible step under an exclusive lock. Three details that matter: - The lock lives in a sidecar `.lock`, so `audit verify` and the console never contend with a decision being recorded. - The tip is read from a trailing window, not the whole file. Re-reading per write would be O(file) and make the audit log the slowest stage in the pipeline; the window grows when a single event exceeds it, and discards a partial leading line rather than parsing a fragment as a record. - A stat() decides whether the cached tip is stale, so the single-writer case pays nothing. p95 is 0.22ms against a 10ms budget, unchanged. `_exclusive` deliberately does not create the parent directory. Doing so turned a write to an unwritable path into a success, which is exactly the failure that must leave the chain tip where it was — caught by an existing contract test. Also adds the approval queue this unblocks: durable, append-only, expiring. Answering mints a proof scoped to the exact held action and verified against it in tests; a proof is never written to disk, because it is single-use and a copy beside the record of what it authorises is a replayable credential. Reading the queue takes the console key; answering takes the admin key on localhost, on the same terms as confirmation/sign. 100% coverage, mypy strict, both benchmarks green, green under CI simulation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ction The queue minted a valid signed proof that authorised nothing. Verified end to end and the retry came back `escalate, allowed=false` — same as with no approval at all. ConfirmationGate wants three pieces of evidence, not one: a token acknowledging the confirmation, a human-readable context, and the proof. Given only the proof it returns early on "evidence is missing" and never looks at it. So an operator would have clicked Approve, seen success, and watched the agent stay blocked. The credential was real and useless. `resolve` now returns every parameter the retry needs, with the token derived from the request id so it points back at the approval that authorised it and the context naming who approved what rather than being filler to clear a length check. Pinned by the pair that matters — unblocked with the evidence, held without it — plus a test that the proof alone still fails, so dropping the token or context from what an approval returns fails loudly instead of quietly reverting. Also: the gateway now runs as a compose service sharing audit-data, so agent traffic and CLI decisions land in one trail and the console shows both. Only safe because of the write lock in the previous commit. And the console JS is checked by a real parser. Two attempts at a pure-Python heuristic both flagged lines the page legitimately contains, so this shells out to `node --check` — preinstalled on CI runners, skipped where absent, with the source-level escape test still covering the class everywhere. Verified to fail on a deliberately broken template: twice now a syntax error stopped the whole page from running while every substring assertion kept passing. 100% coverage, mypy strict, green under CI simulation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.