Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 24 additions & 8 deletions agents/repo-finder.md
Original file line number Diff line number Diff line change
Expand Up @@ -575,7 +575,7 @@ fi
2. A `good first issue` or `help wanted` label (explicit triage-accepted signal).
3. A `kind:*` / `type:*` / `priority:*` / `severity:*` / `area:*` label (maintainer-applied taxonomy — NOT just `needs-triage` / `triage` / `status:need-triage` alone).

Why this gate exists: a `needs-triage`-only issue with zero maintainer comments is an untriaged feature request or unconfirmed bug report. Shipping a PR against it at Apache-/Airflow-class repos triggers a lazy-consensus or devlist redirect that structurally caps merge probability below 95% regardless of code quality. The prior `has_maintainer_approval` field was additive (+3 bonus), so issues without maintainer approval could still reach rank 1 via the other bonuses. This gate converts it to a prerequisite.
Why this gate exists: a `needs-triage`-only issue with zero maintainer comments is an untriaged feature request or unconfirmed bug report. Shipping a PR against it at Apache-/Airflow-class repos triggers a lazy-consensus or devlist redirect that structurally caps merge probability below 95% regardless of code quality. The maintainer-signal bonus is additive, so an issue no maintainer engaged with could otherwise reach rank 1 on the other bonuses alone. This gate converts engagement to a prerequisite.

A bot is not a maintainer — exclude bot accounts from the comment test.

Expand Down Expand Up @@ -726,7 +726,7 @@ jq -r --rawfile m "$MAINTAINERS" \
elif ((now - ($engaged | max)) / 86400 | floor) > $window then
"SKIP \(.number): no maintainer engagement in \(((now - ($engaged | max)) / 86400 | floor))d (window \($window)d) — abandoned"
else
"KEEP \(.number) engaged_days=\(((now - ($engaged | max)) / 86400 | floor)) maintainer_commented=\(.maintainer_commented) approval=\(.maintainer_comment_assoc)"
"KEEP \(.number) engaged_days=\(((now - ($engaged | max)) / 86400 | floor)) maintainer_commented=\(.maintainer_commented) signal=\(.maintainer_signal)"
end
' "$SCRATCH/stage_a.jsonl"
```
Expand All @@ -751,14 +751,30 @@ visibly drop one. Apply the prose here, then say in `notes` that it did.
- **Is a bug** (not a feature request): +3
- **Has reproduction steps**: +2
- **Labeled good-first-issue or help-wanted**: +2
- **Maintainer commented with approval signal** ("PRs welcome", "happy to review"): +3. Requires `maintainer_comment_assoc` — a union-only commenter does not earn this bonus.
- **Maintainer signal**, graded from `maintainer_signal`: `invites_pr` **+5** · `confirms` **+3** · `neutral` **+1** · `none` **0**. Association is the prerequisite, never the grade — a union-only commenter earns nothing here.
- **Maintainer engaged within the last 28 days**: +2
- **Issue age**, graded: 2–30d **+1** · 30–90d **0** · 90–365d **−1** · >365d **−2**. Scored, never hard-skipped — old is riskier, not worthless.
- **Scope is small** (likely < 100 lines, single-file fix): +2

Set `best_issue.has_maintainer_approval` from `maintainer_comment_assoc` — true only when GitHub reported a commenter on this issue as `OWNER`/`MEMBER`/`COLLABORATOR`. Never set it from `maintainer_commented`: that flag includes the wide 3c maintainer union, which is correct for passing the gate but too loose to call approval.

Label-only triage and union-only comments still pass the gate; they just record `has_maintainer_approval=false`. The field answers one question — "did an account GitHub vouches for as a maintainer speak on this issue?" — and the +3 approval bonus is the only thing that reads it.
Set `best_issue.maintainer_signal` from the `maintainer_signal` the triage filter
emitted. Do not recompute it from `maintainer_comment_assoc`: association answers
"did an account GitHub vouches for speak here", which is a prerequisite for being
graded and not a grade. Never derive it from `maintainer_commented` either — that
flag includes the wide 3c union, correct for passing the gate and too loose to
read as endorsement.

The filter grades on comment text, and only on text that is text: it strips links
and bare mentions first, so a maintainer comment that is only a pointer somewhere
else grades `none`. A pointer states no position on this issue.

Read the graded comment yourself before accepting the tier. The filter matches
prose with regexes and will misgrade: a refutation phrased in the project's own
domain terms disputes the premise while matching none of the decline patterns.
**If the maintainer disputes the premise, redirects it, or calls it
working-as-intended, skip the issue outright** rather than scoring it down —
zeroing the bonus still leaves bug + repro + small scope at 7, and labels plus
recency clear the 8 threshold on their own. Record any tier you overrode in
`notes`, so the regex gaps stay visible.

If no issue scores 8+ after applying the hard filter, mark the repo as "no clear opportunity right now" and exclude from the final list.

Expand Down Expand Up @@ -854,7 +870,7 @@ needs that number to know the list is partial rather than the field being thin.
"title": "issue title",
"type": "bug",
"labels": ["bug", "good first issue"],
"has_maintainer_approval": true,
"maintainer_signal": "invites_pr",
"issue_score": 12,
"url": "https://github.com/OWNER/REPO/issues/1234"
},
Expand All @@ -880,7 +896,7 @@ Found N repos worth contributing to (searched M candidates).

## Top Pick
**owner/repo#1234** — [issue title]
- Maintainer approved: yes
- Maintainer signal: invites a PR
- Type: bug
- Estimated scope: small (single file)
- Median PR merge time: 2 days
Expand Down
5 changes: 4 additions & 1 deletion schemas/repo_shortlist.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,10 @@
"title": {"type": "string"},
"type": {"type": "string"},
"labels": {"type": "array", "items": {"type": "string"}},
"has_maintainer_approval": {"type": "boolean"},
"maintainer_signal": {
"type": "string",
"enum": ["invites_pr", "confirms", "neutral", "none"]
},
"issue_score": {"type": "number"},
"url": {"type": "string"}
}
Expand Down
59 changes: 58 additions & 1 deletion scripts/orchestrator/triage_filter.sh
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,31 @@ jq -c --rawfile m "$MAINTAINERS" --argjson now "$NOW" '
+ "|i.m working on|working on (a fix|this)"
+ "|(already |should have )?fixed (it |this )?in [a-z0-9.@/#-]"
+ "|(fix|patch) (has )?landed|landed in|resolved (in|by) (#|[a-z0-9])"
+ "|(fix|pr) (is )?incoming") as $claimed
+ "|(fix|pr) (is )?incoming"
# A claim is often worded as an intention to act rather than a patch in hand.
# "this is valid, we will look into it" confirms the defect and takes it in one
# sentence — which is why the claim test runs before the signal tiering below.
# "on our side" only claims the work when it is attached to an intent to act:
# "the regression is on our side" owns the defect and invites a fix.
+ "|we.ll look into|will look into|looking into (this|it) on our"
+ "|noted (it )?(on our side|internally)|taking (this|it) on") as $claimed

# An outsider saying "I would like to work on this" is not an assignee, so the
# assignee test cannot see the claim. Windowed, unlike the maintainer test: a
# drive-by claim nobody acted on must not fence the issue off forever.
| ("i.d like to work on|i would like to work on|can i (take|work on|pick up)"
+ "|i.ll (take|work on|pick) (this|it)|assign (this|it) to me"
+ "|i.m (going to |gonna )?work(ing)? on (this|it)"
+ "|(may|could) i (take|work on) (this|it)") as $outsider_claim
| 1209600 as $claim_ttl

# Association makes a comment gradeable; it does not make it approval. The text
# sets the grade.
| ("prs? (are )?welcome|welcome a (pr|patch)|happy to (review|take a pr|accept)"
+ "|feel free to (submit|open|send)|would (accept|welcome) a (pr|patch)"
+ "|go ahead and (open|submit)|contributions? welcome") as $invites
| ("reproduc|confirmed|i see the same|can confirm|this is (valid|a bug|indeed)"
+ "|good catch|you.re right|nice find|makes sense to me") as $confirms

# Announcements and containers wear defect labels. A pinned "the project moved"
# notice earns a maintainer taxonomy label and every engagement signal the gate
Expand Down Expand Up @@ -104,6 +128,35 @@ jq -c --rawfile m "$MAINTAINERS" --argjson now "$NOW" '
| ($i.labels | map(.name | ascii_downcase)) as $L
| ($mc | map(.body // "") | join("\n") | ascii_downcase) as $mbody

# Grade only what is left after stripping links and bare @mentions: a comment
# that is only a pointer somewhere else states no position on this issue, and no
# sentiment tier can read one.
| ($mca | map((.body // "")
| ascii_downcase
| gsub("https?://\\S+"; " ")
| gsub("@[a-z0-9_-]+"; " ")
| gsub("\\s+"; " ") | sub("^ +"; "") | sub(" +$"; ""))
| map(select(length > 0))) as $stripped
| ($stripped | join("\n")) as $gbody
# Match before measuring. The floor separates a remark from a pointer, so it
# decides neutral-or-none only — applied first it would grade "PRs welcome!"
# (12 chars, the strongest signal there is) as no signal at all.
| (if ($stripped | length) == 0 then "none"
elif ($gbody | test($invites)) then "invites_pr"
elif ($gbody | test($confirms)) then "confirms"
elif ($stripped | any(length >= 15)) then "neutral"
else "none" end) as $signal

# Non-maintainer claims, still inside the window.
| ($human | map(select((.authorAssociation | IN("OWNER","MEMBER","COLLABORATOR")) | not))
# `// ""` is not enough: an undated comment must not reach
# fromdateiso8601, which throws on null and aborts the whole batch.
# Undated reads as old, so the claim expires rather than fencing off.
| map(select(((.body // "") | ascii_downcase | test($outsider_claim))
and (((.createdAt // "") | length) > 0)
and ((.createdAt | fromdateiso8601) > ($now - $claim_ttl))))
| length) as $outsider_live

# (24h rule) an issue younger than a day has not been triaged; racing to it
# produces noise PRs.
| select(($i.createdAt | fromdateiso8601) <= ($now - 86400))
Expand Down Expand Up @@ -139,6 +192,9 @@ jq -c --rawfile m "$MAINTAINERS" --argjson now "$NOW" '
elif ($mbody | test($claimed)) then
{verdict: "SKIP", number: $i.number,
reason: "maintainer is already fixing it"}
elif $outsider_live > 0 then
{verdict: "SKIP", number: $i.number,
reason: "claimed — another contributor said they are taking it"}
else
# Carry the issue payload, not just a verdict on a number. The Step-4 rubric
# scores title, labels, body and createdAt, and the agent contract forbids
Expand All @@ -150,6 +206,7 @@ jq -c --rawfile m "$MAINTAINERS" --argjson now "$NOW" '
createdAt: $i.createdAt,
maintainer_commented: ($mc | length > 0),
maintainer_comment_assoc: ($mca | length > 0),
maintainer_signal: $signal,
last_maintainer_comment: ($mc | map(.createdAt) | max)}
end
' "$ISSUES"
97 changes: 96 additions & 1 deletion tests/scripts/test_triage_filter.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ trap 'rm -rf "$tmpdir"' EXIT
NOW=1783814400
OLD="2026-07-01T00:00:00Z" # 11 days old — clears the 24h rule
FRESH="2026-07-11T18:00:00Z" # 6 hours old — must be dropped
RECENT="2026-07-10T00:00:00Z" # 2 days old — a claim this fresh is still live
STALE="2026-05-20T00:00:00Z" # 53 days old — a claim this old is abandoned
Comment thread
coderabbitai[bot] marked this conversation as resolved.

printf 'unionlead\n' > "$tmpdir/maintainers.txt"

Expand Down Expand Up @@ -106,7 +108,59 @@ cat > "$tmpdir/issues.json" <<EOF
{"number": 913, "title": "[v4] Loader raises on a valid checkpoint",
"createdAt": "$OLD", "assignees": [], "labels": [{"name": "bug"}],
"comments": [{"authorAssociation": "MEMBER", "author": {"login": "maint-b"},
"body": "We can't fix bugs on v4 — please upgrade to v5.", "createdAt": "$OLD"}]}
"body": "We can't fix bugs on v4 — please upgrade to v5.", "createdAt": "$OLD"}]},

{"number": 914, "title": "Indexer produces incomplete examples for one doc format",
"createdAt": "$OLD", "assignees": [], "labels": [{"name": "bug"}],
"comments": [{"authorAssociation": "COLLABORATOR", "author": {"login": "maint-d"},
"body": "Thanks for the report, this is valid. We have noted it on our side and will look into supporting that format.",
"createdAt": "$OLD"}]},

{"number": 915, "title": "Batched decode corrupts every row after the first",
"createdAt": "$OLD", "assignees": [], "labels": [{"name": "kind:bug"}],
"comments": [{"authorAssociation": "NONE", "author": {"login": "eager-newcomer"},
"body": "Hi! I'd like to work on this issue if it's still available.",
"createdAt": "$RECENT"}]},

{"number": 916, "title": "Same claim, long abandoned",
"createdAt": "$OLD", "assignees": [], "labels": [{"name": "kind:bug"}],
"comments": [{"authorAssociation": "NONE", "author": {"login": "eager-newcomer"},
"body": "Hi! I'd like to work on this issue if it's still available.",
"createdAt": "$STALE"}]},

{"number": 917, "title": "Usage totals are summed across tool-loop calls",
"createdAt": "$OLD", "assignees": [], "labels": [{"name": "bug"}],
"comments": [{"authorAssociation": "COLLABORATOR", "author": {"login": "maint-d"},
"body": "https://example.invalid/acme/repo/pull/1", "createdAt": "$OLD"}]},

{"number": 918, "title": "Quantized matmul ignores the leading batch dimension",
"createdAt": "$OLD", "assignees": [], "labels": [{"name": "bug"}],
"comments": [{"authorAssociation": "MEMBER", "author": {"login": "maint-b"},
"body": "cc @maint-c for the quantization path", "createdAt": "$OLD"}]},

{"number": 919, "title": "Join is unsanitised on the secondary path",
"createdAt": "$OLD", "assignees": [], "labels": [{"name": "bug"}],
"comments": [{"authorAssociation": "MEMBER", "author": {"login": "maint-b"},
"body": "Reproduced on main — the second join has the same defect.",
"createdAt": "$OLD"}]},

{"number": 920, "title": "Terse blessing is still a blessing",
"createdAt": "$OLD", "assignees": [], "labels": [{"name": "bug"}],
"comments": [{"authorAssociation": "MEMBER", "author": {"login": "maint-b"},
"body": "PRs welcome!", "createdAt": "$OLD"}]},

{"number": 921, "title": "Owning the defect is not claiming the fix",
"createdAt": "$OLD", "assignees": [], "labels": [{"name": "bug"}],
"comments": [{"authorAssociation": "MEMBER", "author": {"login": "maint-b"},
"body": "Thanks — reproduced. The regression is on our side, not in your config.",
"createdAt": "$OLD"}]},

{"number": 922, "title": "Comment with no timestamp must not abort the batch",
"createdAt": "$OLD", "assignees": [], "labels": [{"name": "bug"}],
"comments": [{"authorAssociation": "NONE", "author": {"login": "eager-newcomer"},
"body": "I would like to work on this."},
{"authorAssociation": "MEMBER", "author": {"login": "maint-b"},
"body": "Reproduced, the parser drops the last byte.", "createdAt": "$OLD"}]}
]
EOF

Expand Down Expand Up @@ -252,4 +306,45 @@ esac
[ "$(field 21795 title)" = "null" ] \
|| { echo "FAIL #21795 is a SKIP and must not carry a title"; exit 1; }

# A maintainer can confirm a defect and claim it in the same breath. Grading the
# sentiment first scores it as approval and never reaches the claim, so the claim
# test must run before the tiering.
want 914 SKIP "maintainer confirmed it and claimed it"
want_reason 914 already

# A non-maintainer claim is not an assignment, so the assignee test cannot see it.
want 915 SKIP "outside contributor claimed it recently"
want_reason 915 claimed

# ...but a claim nobody acted on must not fence the issue off forever.
want 916 KEEP "stale claim is abandoned, not competing"

# Signal tiers. Association is the prerequisite; the comment text sets the grade.
[ "$(field 903 maintainer_signal)" = "invites_pr" ] || { echo "FAIL #903 signal: got '$(field 903 maintainer_signal)'"; exit 1; }
[ "$(field 911 maintainer_signal)" = "invites_pr" ] || { echo "FAIL #911 signal: got '$(field 911 maintainer_signal)'"; exit 1; }
[ "$(field 919 maintainer_signal)" = "confirms" ] || { echo "FAIL #919 signal: got '$(field 919 maintainer_signal)'"; exit 1; }
[ "$(field 918 maintainer_signal)" = "neutral" ] || { echo "FAIL #918 signal: got '$(field 918 maintainer_signal)'"; exit 1; }

# A comment with no prose cannot be graded as prose. A bare link is a pointer,
# not an endorsement, and no sentiment tier can read one.
want 917 KEEP "bare link is not a defect signal"
[ "$(field 917 maintainer_signal)" = "none" ] || { echo "FAIL #917 signal: got '$(field 917 maintainer_signal)'"; exit 1; }

# A union-only lead still earns no signal, whatever they wrote.
[ "$(field 904 maintainer_signal)" = "none" ] || { echo "FAIL #904 signal: got '$(field 904 maintainer_signal)'"; exit 1; }

# The strongest signal is also the shortest one anyone writes. A prose floor
# applied before the match would grade this as no signal at all.
want 920 KEEP "terse invite"
[ "$(field 920 maintainer_signal)" = "invites_pr" ] || { echo "FAIL #920 signal: got '$(field 920 maintainer_signal)'"; exit 1; }

# "on our side" attached to the defect owns the bug; attached to the work claims
# it. Only the second is a skip.
want 921 KEEP "maintainer owns the defect without claiming the fix"
[ "$(field 921 maintainer_signal)" = "confirms" ] || { echo "FAIL #921 signal: got '$(field 921 maintainer_signal)'"; exit 1; }

# A comment can serialize without createdAt, and `null | fromdateiso8601` throws,
# which would abort the batch and take every other issue with it.
want 922 KEEP "undated comment does not abort the batch"

echo "OK test_triage_filter.sh"
Loading