Skip to content

swe-router 0.2.0: route on a six-question rubric - #227

Merged
aarora79 merged 1 commit into
mainfrom
swe-router-0.2.0
Oct 14, 2026
Merged

aarora79 merged 1 commit into
mainfrom
swe-router-0.2.0

Conversation

@aarora79

Copy link
Copy Markdown
Owner

Re-lands #226 on main. #226 merged into its stacked base branch (bench/cli-cli-set-a) before #225 merged, so GitHub never retargeted it and its changes did not reach main.

This branch applies the same diff to main; the resulting tree is identical to #226's reviewed head (af3eb85). See #226 for the summary, the measured effect and the review.

Test plan

The changes reviewed and approved in #226, which merged into its stacked
base branch rather than main. The tree is identical to #226's head.

The skill answers the six questions in rubric.md about the task, and
route.py turns the answers into a difficulty bucket (from four answers)
and a required quality (from blast radius and verification), then picks
the cheapest permitted model whose mean benchmark score in that bucket
clears it. It skips any model that failed a benchmark task in the
bucket, reads the next harder bucket when the benchmark has none for the
task's, and keeps the developer on their model when confidence is low.
--tier/--floor still work for one release.

models.json moves to schema 2.0 with score_by_difficulty and
completion_by_difficulty, built from the v2 Opus 5.5 rubric pass.
rubric.md and routing-policy.json move into vend/swe-router/; the
installer ships both, keeps an edited policy on upgrade, and refuses to
install without python3 3.8+. SKILL.md names no measured model. The
workflow slide, its PNG, the ASCII diagram and both READMEs describe the
new flow, and docs/swe-router-how-it-works and
docs/swe-router-rubric-scores explain it.

On the 21 v2 tasks against claude-opus-5 on everything: 74.7% saved for
-3.22 points and 4 tasks below the bar, where the previous skill gave
70.5%, -5.06 and 9.
@aarora79 aarora79 added the enhancement New feature or request label Oct 14, 2026
@aarora79
aarora79 merged commit 85c6a09 into main Oct 14, 2026
5 checks passed
@aarora79 aarora79 mentioned this pull request Oct 15, 2026
3 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant