-
Notifications
You must be signed in to change notification settings - Fork 41
Review agent: suppress content-security URL findings in documentation files #1629
Copy link
Copy link
Open
Labels
agent/retroRetro agentRetro agentagent/reviewReview agentReview agentcomponent/harnessAgent harness, config, and skills loadingAgent harness, config, and skills loadingfeatureFeature-category issue awaiting human prioritizationFeature-category issue awaiting human prioritizationpriority/mediumNormal priority, plan for next cycleNormal priority, plan for next cycletriagedTriaged but awaiting human prioritizationTriaged but awaiting human prioritizationtype/featureNew capability requestNew capability request
Metadata
Metadata
Assignees
Labels
agent/retroRetro agentRetro agentagent/reviewReview agentReview agentcomponent/harnessAgent harness, config, and skills loadingAgent harness, config, and skills loadingfeatureFeature-category issue awaiting human prioritizationFeature-category issue awaiting human prioritizationpriority/mediumNormal priority, plan for next cycleNormal priority, plan for next cycletriagedTriaged but awaiting human prioritizationTriaged but awaiting human prioritizationtype/featureNew capability requestNew capability request
Type
Fields
Give feedbackNo fields configured for issues without a type.
Projects
Status
Todo
What happened
On PR #1483, the review agent flagged a Cloud Run URL in
docs/guides/admin/install.mdas[high] content-security, recommending it be replaced with an environment variable. The URL was intentionally hardcoded — the install guide needed to show users the actual endpoint to configure. The human author replied explaining this, but the agent re-flagged the same URL on the next review pass. This false positive consumed author attention across two review cycles.What could go better
The content-security heuristic that detects hardcoded URLs/secrets does not distinguish between executable code (where hardcoded credentials are a real risk) and documentation (where showing actual URLs is often the point). A URL in a markdown file under
docs/cannot be exploited the same way as one in source code — it is descriptive, not operative. The review agent should calibrate its content-security scanning based on file type: markdown/docs files should have a much higher bar for URL-based security findings. Confidence: high — this is a clear category error in the heuristic. The agent correctly identifies that hardcoded URLs can be a security concern, but misapplies that rule to documentation where hardcoded URLs are expected and appropriate.Proposed change
In the
code-reviewskill (internal/scaffold/fullsend-repo/skills/code-review/SKILL.md), add guidance to the content-security evaluation dimension: when the file under review is documentation (markdown, RST, or similar non-executable format underdocs/), URLs and service endpoints should not be flagged as content-security findings unless they contain obvious secrets (tokens, API keys, passwords). Hardcoded URLs in documentation are expected and should be treated differently from hardcoded URLs in source code. This could be implemented as a file-type-aware severity modifier: content-security findings about URLs indocs/**/*.mdfiles should be capped at[info]severity unless the URL contains path segments that look like secrets.Validation criteria
On the next 5 documentation-only PRs reviewed by the review agent, none should receive a
[high]or[medium]content-security finding solely for containing a URL in a markdown file. If a docs PR does contain an actual secret (API key, token), it should still be flagged. Timeframe: next 30 days or 5 docs PRs, whichever comes first.Generated by retro agent from #1483