Skip to content

feat(auto-permissions): count confirmed dialog answers as user evidence - #25

Draft
hank-warren wants to merge 2 commits into
ogulcancelik:mainfrom
hank-warren:feat/review-evidence-user-answer-tools
Draft

feat(auto-permissions): count confirmed dialog answers as user evidence#25
hank-warren wants to merge 2 commits into
ogulcancelik:mainfrom
hank-warren:feat/review-evidence-user-answer-tools

Conversation

@hank-warren

@hank-warren hank-warren commented Aug 6, 2026

Copy link
Copy Markdown

Implements #24: an operator-allowlisted, tool-agnostic way for confirmed
interactive-dialog answers to count as user authorization evidence.

Changes

  • config.tsreviewEvidence.userAnswerTools: string[] (default [],
    trimmed, validated; current behavior unchanged when absent).
  • review.tscollectReviewEvidence takes the allowlist. A successful,
    non-cancelled toolResult from an allowlisted tool (matched on base name, so
    functions.ask_user_question qualifies) whose details carry
    { answers: [{ question, answer? | selected[], notes? }], cancelled: false }
    (and no error field) emits one record per answered question at the
    toolResult entry:
    { source: "user", text: 'USER (dialog answer): selected "…" — assistant-drafted question: "…"' }.
    The user-chosen content leads the record; the assistant-drafted question is
    labeled as quoted context. Bare allowlist names match the tool in any
    namespace; dotted names match exactly.
    Keys are stable (entryId:answerIndex:dialog-answer), so delta mode and the
    evidence-prefix cache are unaffected. Envelope prose, cancelled/errored
    results, and unanswered questions are never read.
  • System prompt — one sentence defining USER (dialog answer): records:
    authorization for exactly the selected content, with the caveat that question
    and option wording is assistant-drafted.
  • index.ts — passes the allowlist into evidence collection and adds it to the
    reviewer fingerprint so a policy change resets reviewer lineage.
  • README — documents the option and the trust reasoning (allowlist as explicit
    operator trust in the tool name; nothing inferred).

Testing

  • bun test: 50 pass (12 new — config validation/trim/reject, promotion,
    multi-select + notes, base-name matching, and the full ignore matrix:
    non-allowlisted, cancelled, errored, empty, malformed).
  • Replayed the real session from reviewEvidence: option to count confirmed interactive-dialog answers as user authorization #24 through the patched collector with
    userAnswerTools: ["ask_user_question"]: the dialog selections surface as
    USER (dialog answer): records naming the exact files and commit message the
    guardian previously could not see. The same evidence, injected via a bridge
    extension, was verified end-to-end to flip the guardian from ask_user to
    approve on that command.

Draft pending your take on the shape from #24 — happy to adjust naming, record
format, or the prompt wording.


Update: after two independent review passes, pushed a hardening commit
(2a5269f): verbatim allowlist matching (a dotted entry can pin an exact tool;
bare names match across namespaces), strict cancelled: false / boolean
isError gating, notes-only answers rejected, records restructured to lead
with the user-chosen content, the prompt rule rooted in the structured source
field, fingerprint/config normalization, an end-to-end harness test proving the
allowlist threads through real config load and resets reviewer lineage, and the
full accept/reject contract documented in the README. 52 tests pass.

- allowlist entries match verbatim; bare names match any namespace, dotted names are exact
- require explicit cancelled:false and boolean isError; reject notes-only answers
- lead records with the user-selected content and label the assistant-drafted question
- root the prompt rule in the structured source field and mark the question as non-instruction
- fingerprint: omit empty allowlist, sort entries; config: deduplicate
- add end-to-end harness test threading config into evidence and resetting lineage
- document the full accept/reject contract and matching semantics; extend security boundary
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant