Record whether the scope filter or the optimizer declined an injection - #50
Merged
Merged
Conversation
`security_domain_filter` answers an out-of-scope controllable event with a `ControllableNoInjection` without consulting the optimizer, and `trajectory_recorder` sits outside it, so both the event and the decline are recorded. An optimizer that is consulted and passes produces a byte-identical entry. On disk the two are indistinguishable. They are opposite findings. "The attacker had no write access to this surface" belongs in a not-applicable bucket; "the attacker had access and chose not to use it" is an attacker result. Only the second says anything about the attacker, and a zero-injection analysis that conflates them reports the framework's own scope policy as attacker behaviour. This is not hypothetical. A DTAP audit read a results tree, counted every recorded controllable event as an offer the attacker had declined, and concluded 41 tasks showed a classifier defect. Recomputed with the scope taken into account, 35 of them never had an in-scope surface fire at all: the agent was calling `terminal` tools under a grant that covered only `gmail`/`slack`. The finding was off by an order of magnitude and the fix aimed at the wrong layer. Nothing on disk could have caught it. Add `declined_by: Literal["scope", "optimizer"]` to `ControllableNoInjection`, set it in the filter, and persist it. The default is `"optimizer"`, which is correct for every optimizer-constructed response, so no module changes and no existing behaviour changes: 589 pre-existing tests pass untouched.
The record gained a field while `SCHEMA_VERSION` stayed at 4, so a reader has
no way to tell a tree that records who declined from one that does not. Every
existing tree lacks the key, and the obvious way to read it,
`entry.get("declined_by", "optimizer")`, silently re-commits the exact
conflation the field exists to prevent: it reports the framework's own scope
policy as attacker behaviour.
Absence of the key means UNKNOWN, never "optimizer". Version 5 says so, and the
constant carries that rule as a comment where a reader will meet it.
The two tests that hardcoded `== 4` now assert against `SCHEMA_VERSION`, so a
future bump does not require editing them.
The PR bumps SCHEMA_VERSION to 5 but reference/results.md still said version 4, which is the page a results-tree consumer reads to learn which fields a persisted trajectory carries. Also names what 5 adds and repeats the read rule that absence of declined_by means unknown, never optimizer.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
security_domain_filteranswers an out-of-scope controllable event with aControllableNoInjectionwithout consulting the optimizer (core/middleware.py:99-110).trajectory_recordercomposes outside it, so the event and the decline are both recorded. An optimizer that is consulted and passes produces a byte-identical entry.On disk the two are indistinguishable:
{"kind":"response","type":"ControllableNoInjection","controllable":"env_tool:terminal.system"} {"kind":"response","type":"ControllableNoInjection","controllable":"env_tool:gmail.public"}The first is "the attacker had no write access here". The second is "the attacker had access and chose not to use it". Those are opposite findings, and only the second says anything about the attacker. The first belongs in a not-applicable bucket, never in a defence rate.
Why this is worth a public field
It is not hypothetical. A DTAP audit read a results tree, counted every recorded controllable event as an offer the attacker had declined, and concluded that 41 tasks showed an attacker-side classifier defect.
Recomputed with the scope taken into account (inclusion is prefix-based on the dotted tag path, so
tools.terminalincludestools.terminal.system):Thirty of those tasks had the agent calling
terminaltools under a grant covering onlygmail/slack. The finding was off by roughly an order of magnitude, and the remedy was aimed at the wrong layer. Nothing on disk could have caught it, which is what this PR changes.The type's own docstring was part of the trap: it described only the framework case ("Returned by the controller when an event's controllable falls outside the security domain tag being tested"), even though every optimizer also returns this to decline.
The change
ControllableNoInjectiongainsdeclined_by: Literal["scope", "optimizer"] = "optimizer".security_domain_filtersets"scope"on its own declines._serialize_responsepersists the field, so it reaches the results tree.reference/events-and-trajectory.mdnow tells analysts to read the field before interpreting any decline.Backward compatible by construction. The dataclass is
kw_only=True, so no positional construction exists to break, and the default is correct for every optimizer-built response. There is exactly one other construction site in the framework (the filter itself). Acrosssuperred-modulesthere are ~200, all keyword-constructed, all correctly defaulting to"optimizer". Nothing in the framework hashes or compares responses, so the added field is inert for equality.load_trajectoryreturns raw JSON, so the read-back path needs no change.Verification
Round-tripped end to end through the real filter and the real serializer:
591 passed(589 pre-existing untouched, plus 2 new) -ruff check/formatclean -mypyclean on 25 source files.Two tests pin the distinction: the filter marks its own declines
"scope", and a handler-constructed decline stays"optimizer".