Repository navigation
Fix/lru eviction and effective fingerprint - #4
Merged
Merged
Conversation
The dashboard was a form per endpoint -- fine for poking the API, useless for what phase 1 actually requires: looking at what the engine decided and reviewing every hard_deny by hand before enforcement goes on. That work was CLI-only. The console shows a live decision feed with per-state filtering and a distribution bar, the phase-readiness criteria with their verdict, and the hard-deny events laid out for review. Two read-only endpoints back it: /v1/decisions and /v1/rollout/report. It lives inside AETHERYA rather than in a separate frontend because this page exposes every decision and every action an agent proposed -- the most sensitive data in the system. A separate app would need its own deployment, its own auth and its own path to the audit trail: three new things to protect to show the same data. Server-rendered with no build step, it ships in the container that is already running and keeps the trail behind one boundary. The API stays open, so a richer frontend can still be built on top without touching the engine. Gated behind AETHERYA_CONSOLE_API_KEY when set, open otherwise like the rest of the decision profile; the page and the route comment both say the port must not be public. Removed _dashboard_html (~11KB of unreachable HTML) rather than leaving it to rot next to its replacement, and dropped the test that asserted its form controls -- test_console.py covers the new page. Verified against the running stack: rebuilt, healthy, and the console renders real traffic across log_only, deny and hard_deny. 1194 tests, 100% coverage under CI's dependency set, mypy strict, ruff and black clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The sidecar client requires editing the agent loop. Most agent runtimes — OpenClaw, LiteLLM, anything that accepts a custom OpenAI-compatible provider — cannot be edited that way. This gates them by changing what the agent talks to instead of what it does: the gateway speaks chat-completions, forwards upstream to OpenAI or Anthropic, and rules on every proposed tool call before the agent sees it. Integration is a config change; OpenClaw is not touched. A refused call is removed from the response and replaced with a line of text explaining why, so the model can replan. A silently dropped call just gets retried. Notable decisions: - Gating requires buffering. A verdict cannot be formed from half a tool call, so `stream: true` still returns SSE frames but emits them after the verdict. The protocol survives; token-by-token latency does not. Documented as a cost, not hidden. - Streamed tool calls carry `index`. Clients reassemble by it; two calls emitted without one merge into a single malformed call. - An actor absent from the capability matrix is denied on every call, and that is invisible until phase 3 because `deny` is not enforced before it. The gateway refuses to start at phase 3 with an unknown actor and warns otherwise. - SDKs are imported by name, following llm_provider: both are optional extras, and a module-level import would make them mandatory for type-checking and for deployments using the other provider. - Malformed tool arguments are gated as raw text rather than dropped. Dropping them would skip the check on exactly the input most likely to be evasive. - Escalations are refused at phase 3, not held: a single chat-completions call has no approval round trip. Asserted in tests so the doc cannot drift. Verified: the real OpenAI SDK parses the gateway's output in both streaming and non-streaming modes, and a refused call does not reach the client. The live upstream path needs an API key and is exercised by scripts/gateway_smoke.py, which is not part of CI. 100% coverage, mypy strict + ruff + black clean, and green under a CI simulation with the optional SDKs absent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The console told you "10 hard_deny events require manual review" and gave you no way to record that you had. `hard_deny_reviewed` was hardcoded to False, so the phase gate could never pass from the UI and the criterion was decorative. This makes it real: verdicts are recorded against the audit `event_id` and the criterion passes on those, and only those. The store is append-only. A review is what unblocks enforcement, so who said what and when has to survive; re-reviewing appends and the earlier verdict stays on disk. Latest wins on read. Three things deliberately do not pass the gate: - Any `false_positive`. Reviewing everything is not approving it — enforcing a rule a human already called wrong is the exact outcome phase 1 exists to prevent. It also requires a note, because the note is what identifies the rule to narrow later. - Events beyond `max_hard_deny_samples`. The report samples; reviewing what was sampled must not clear a gate covering events nobody was shown. - A verdict on an id that is not a hard_deny, or not in the trail at all. It would be recorded, counted by nothing, and leave the operator believing they had reviewed something. Fixes a bug in the criterion I introduced with it: a window with zero hard_deny also never passed, so a well-behaved agent could never leave phase 1. Nothing to judge is not the same as a judgement withheld. Recording a review is the console's only write and is never open, even when the reads are: without a key the verdict carries a reviewer name nobody had to prove, which is worse than no attribution at all. Verified end to end against a running server: the criterion holds at 2 of 3, flips to PASS at 3 of 3, and a later false positive drops it back to HOLD with the earlier verdict still in the history. CLI and console agree via `rollout report --review-path`. 100% coverage, mypy strict + ruff + black clean, green under CI simulation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.