Repository navigation
fix(review-utils): let the reviewer write its metrics file - #369
Conversation
Claude Code permission rules treat a path with a single leading `/` as
relative to the settings source, so `Write(/home/runner/work/_temp/**)`
never matched. Every Write was denied, review-metrics.yaml was never
written, and the receiver's metrics gate failed (agent-jordan#2).
- Use the absolute `//` form (`Write(/${{ runner.temp }}/**)`) and add the
matching Edit rule, in both --allowedTools and settings.permissions.allow.
- Log permission denials from the execution file as warnings and in the
step summary so the next mis-scoped rule is visible.
- SKILL.md: name the notes file location, drop the empty ${REVIEW_URL}
placeholder, describe the Write rule correctly, and point at the
inlined partial sections instead of file paths.
- templates/dispatch-receiver-review.yaml: match the real receiver
(event types the decider actually sends, including opened/reopened;
the secret names review-receiver.yaml requires).
- docs/research: add the as-is audit of the review pipeline.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012K2oct3va5FBB9tW7CNCpE
Plugin Version PreviewPreview only — plugin versions and
|
There was a problem hiding this comment.
✅ LGTM — targeted fix with the right evidence trail
- ✅
//-form absolute-path fix is correct and well-explained; empirical evidence (run 37557540105, agent-jordan#2) backs it up
- ✅ New permission-denials step turns previously-silent failures into visible warnings — net-positive debug infra
- ✅ Template now matches
review-receiver.yaml's actual secrets: + inputs: contract (event types, secret names, consumer_workflow_run_url key)
- ✅ SKILL.md edits align the agent's instructions with the new permission scope and drop a dead
${REVIEW_URL} placeholder
- ❔ One inline note on the
gsub pattern (works, but reads like a no-op), one on the dead trigger-event-name receiver input (predates this PR)
Click to expand for full details
//-form absolute-path fix is correct and well-explained; empirical evidence (run 37557540105, agent-jordan#2) backs it upreview-receiver.yaml's actual secrets: + inputs: contract (event types, secret names, consumer_workflow_run_url key)${REVIEW_URL} placeholdergsub pattern (works, but reads like a no-op), one on the dead trigger-event-name receiver input (predates this PR)How I verified
The // fix (action.yaml). I didn't find a quotable public Claude Code docs line stating the single-vs-double-slash rule, but the PR cites a real failing run (37557540105) and a concrete symptom chain (every Write denied → metrics never written → receiver's metrics gate failed on agent-jordan#2). The rendered form //home/runner/work/_temp/** matches the standard "//abs/path" absolute-path convention several tools use for distinguishing project-relative from filesystem-absolute paths. The matching settings.permissions.allow entries use the same form. The old single-slash form in --allowedTools and settings.permissions.allow is replaced in both places. Edit is scoped the same way as Write, and the temp dir was already in additionalDirectories, so blast radius is "same dir we already shared, now with Write+Edit instead of just Read."
The permission-denials step. I pulled anthropics/claude-code-action/base-action/src/run-claude-sdk.ts and execution-file.ts:
- The execution file (
$RUNNER_TEMP/claude-execution-output.json) isJSON.stringify(messages)— a raw array of SDK messages. ✅ - The result message exposes
resultMsg.permission_denialsas an array (the summary code readsresultMsg.permission_denials?.length ?? 0). ✅
So the step's filter .[] | select(.type == "result") | .permission_denials // [] | .[] is correct.
The gsub escape. On first read it looks like a no-op (gsub("\\|"; "\\|")), but working through jq semantics:
- Pattern
"\\|"is a jq string\|, which in PCRE regex matches a literal|. - Replacement
"\\|"is a jq string\|(jq's gsub replacement treats\\as a literal\, and|has no special meaning), so the output is the 2-char\|. - Net effect: raw
|intool_input→\|in the markdown cell. Correct, just confusing — see the inline comment.
The template changes. I compared templates/dispatch-receiver-review.yaml against review-receiver.yaml's on: workflow_call schema at main:
- Required secrets
AUTOMATION_GITHUB_APP_ID/KEY,REVIEW_GITHUB_APP_ID/KEY— ✅ matched. - Optional LLM secrets
ANTHROPIC_API_KEY,CLAUDE_CODE_OAUTH_TOKEN— ✅ matched (previously the template hadREVIEW_ANTHROPIC_API_KEY, which the receiver would not accept). - Inputs — ✅
trigger-event-name,trigger-event-action,consumer-workflow-run-urlall map to receiver inputs that exist. - Event-type list — the decider sends
event-type: ${{ github.event_name }}/${{ github.event.action }}(fromreview-dispatch.yaml), sopull_request/opened|reopened|synchronize|ready_for_review|labeled|converted_to_draftis the right set. The oldpr-reviewtype is gone from both sides.
One observation on the receiver side (not this PR): trigger-event-name is declared as an input but never read (only trigger-event-action is used in the skip/review job gates), and its description is a copy-paste of trigger-event-action's. Called out in the inline comment on the template.
The research doc. 662 lines of as-is pipeline audit. No workflow references it, so it's docs-only. Not a load-bearing part of the fix.
Scoring rationale
- Quality 92%: Clean, targeted change. Clear WHY in the diff comments and PR body. Validation plan is specific (extract the step's script, run it against a sample execution file, re-run review end-to-end to prove the
//fix). The one subtlegsubdeserves a one-line comment. - Security N/A: Nothing security-relevant beyond widening Write/Edit to the runner temp dir, which is already shared via
additionalDirectoriesand ephemeral per-job. - Simplicity 88%: The permission-rule semantics are inherently subtle and the comment block earns its length. The permission-denials step is small and self-contained.
- Confidence 90%: High on
//-fix + template + SKILL.md diff (verified against the receiver's workflow_call contract and against claude-code-action's own code). Can't be 100% without seeing the next review-run execution file showPermission denials: 0.
Follow-ups:
- P2: Add a one-line comment next to the
gsubso a future reader isn't fooled into thinking the escape is a no-op (inline suggestion). - P2:
review-receiver.yamltrigger-event-nameis a dead input with misleading description — wire it up or drop it (follow-up PR, not this one). - P2: After the next review run, confirm the step summary shows
Permission denials: 0(or at least noWrite/Editdenials) — that's the live validation this PR couldn't do.
workflow run · anthropics/claude-code-action base-action/src/run-claude-sdk.ts · anthropics/claude-code-action base-action/src/execution-file.ts
| echo | ||
| echo '| Tool | Input |' | ||
| echo '| --- | --- |' | ||
| jq -r '.[] | "| `\(.tool_name)` | `\(.tool_input | tostring | .[0:200] | gsub("\\|"; "\\|"))` |"' <<<"$denials" |
There was a problem hiding this comment.
✅ The gsub("\\|"; "\\|") escaping works correctly despite looking like a no-op: in jq, the pattern "\\|" is PCRE regex matching a literal |, and the replacement "\\|" is a two-char literal \|. So raw | in tool_input becomes \| in the markdown table cell — correct.
Worth a one-line comment though — a future reader will almost certainly flag this as suspicious.
| jq -r '.[] | "| `\(.tool_name)` | `\(.tool_input | tostring | .[0:200] | gsub("\\|"; "\\|"))` |"' <<<"$denials" | |
| jq -r '.[] | "| `\(.tool_name)` | `\(.tool_input | tostring | .[0:200] | gsub("\\|"; "\\|"))` |"' <<<"$denials" # gsub escapes raw `|` -> `\|` for markdown table cells |
| consumer-workflow-run-url: ${{ github.event.client_payload.consumer_workflow_run_url }} | ||
| # Forward the consumer PR's event action so the receiver can short-circuit | ||
| # on `converted_to_draft` (posts neutral check + exits, no review run). | ||
| trigger-event-name: ${{ github.event.client_payload.trigger.event }} |
There was a problem hiding this comment.
❔ The template forwards client_payload.trigger.event as trigger-event-name to review-receiver.yaml, which is correct. But in the receiver (.github/workflows/review-receiver.yaml lines 74–82) the input's description is a copy-paste of trigger-event-action ("The action field from the consumer PR event…"), and the receiver never references inputs.trigger-event-name anywhere — only trigger-event-action is read (lines 115/156). So trigger-event-name is currently a dead input with misleading docs.
Not this PR's problem (the input predates it), but worth a follow-up to either wire the receiver to it or drop the input entirely.
What changed
plugins/claude-code/review-utils/actions/run-agent/action.yamlWrite(/${{ runner.temp }}/**), which renders as//home/runner/work/_temp/**. A matchingEditrule is added. Both rules go in--allowedToolsand insettings.permissions.allow. The comments are rewritten to explain the//requirement.permission_denialsfrom claude-code-action'sexecution_fileand emits one::warningper denial plus a step-summary table.plugins/claude-code/review-utils/skills/review-code/SKILL.md${REVIEW_METRICS_PATH}. The text describing the Write rule is corrected. The example's${REVIEW_URL}(never set, so envsubst made it empty) is replaced. The partials are now referred to by their inlined section names instead of file paths.templates/dispatch-receiver-review.yamlpull_request/<action>, now includingopenedandreopened) instead ofpr-review. It passes the secret namesreview-receiver.yamlrequires and reads theconsumer_workflow_run_urlkey.docs/research/review-pipeline-as-is-2026-10-07.mdWhy
The review format on nsheaps/agent-jordan#2 was wrong. #368 fixed that by inlining the partials. The run behind that PR (37557540105) also failed its metrics gate with "No metrics file":
/means a path relative to the settings source, not the filesystem root. An absolute path needs//.Write(/home/runner/work/_temp/**)never matched, and every Write to the temp dir was denied, silently.Blast radius
review-receiver.yaml, becauseplugin-refdefaults tomain. Henry's receiver pins the workflow by SHA but checks out the plugin atmain.additionalDirectories.Validation performed
I extracted the new step's script from the YAML and ran it against a sample execution file:
The
//fix itself can only be proven by a live review run. The success signal is a run with a metrics file andPermission denials: 0(or no Write denials).What a reviewer should scrutinise
//absolute-path semantics. This comes from Claude Code's permission docs. If the action's settings path is handled differently, the new denial log will show it on the next run.Editin the temp dir is slightly wider than strictly needed. It lets the agent update its notes file in place.Related
opened/reopenedtriggers.🤖 Generated with Claude Code
https://claude.ai/code/session_012K2oct3va5FBB9tW7CNCpE
Generated by Claude Code