Skip to content

Issue scoring system: WSJF-derived rubric + in-issue olo-score tooling - #466

Merged
drsnuggles8 merged 2 commits into
masterfrom
feature/issue-scoring-system
Jun 29, 2026
Merged

drsnuggles8 merged 2 commits into
masterfrom
feature/issue-scoring-system

Conversation

@drsnuggles8

Copy link
Copy Markdown
Owner

What

A lightweight, explicit system for rating issues/tasks, replacing gut-feel backlog ordering. Derived from WSJF / Cost of Delay, retuned for a personal game engine (no commercial "business value"; the backlog is a tech tree; rendering can look done while being wrong).

The rubric — docs/process/issue-scoring.md

  • Value axes (Cost of Delay): Capability, Craft, Stability, Decay — Fibonacci {1,2,3,5,8,13}.
  • Effort denominator, calibrated against real PR velocity (typical task = 1; CI/review latency is not effort).
  • Confidence multiplier (0.5 = spike-first — the "tests green, screen wrong" guard).
  • Personal axes (Learning, Fun), kept out of the value math; Fun ≥ 8 triggers a logged Pull-override.
  • Kano (table-stakes / performance / delighter) + tech-tree edges (blocked_by / blocks) as tags.
  • Score = Confidence × (Capability + Craft + Stability + Decay) / Effort. Anchors locked in §4.

Storage = the issues themselves

Each issue carries a visible ## Score section with an olo-score fenced block holding only the raw inputs — single public source of truth, born with the issue, can't rot out of sync with a side file. Nothing derived (score/rank/tier) is stored. All open issues have been seeded.

Tooling — scripts/issue_scores.py

  • rank — derive scores live from the issues, apply the Pull-override, print the ranked list (what /start-work calls).
  • lint — flag any unscored open issue.
  • apply — migration helper (seeds blocks from a local source).

Born-scored

Both issue templates (feature_request, bug_report) now ship a neutral olo-score block.

Note

The /start-work and /pr-status workflow-loop skills were rewired to select/order by the score, but those live in user-level ~/.claude/commands (not this repo), so they're not in this diff.

🤖 Generated with Claude Code

drsnuggles8 and others added 2 commits June 29, 2026 11:21
WSJF/Cost-of-Delay-derived rubric tuned for this engine: Capability,
Craft, Stability, Decay over Effort (× Confidence), plus personal
Learning/Fun axes and Kano + tech-tree tags. Scores live in an
`olo-score` block in each issue body (single public source of truth);
nothing derived (score/rank/tier) is stored — `scripts/issue_scores.py`
ranks on demand (rank/lint/apply). Feature issue template seeds a
neutral block so new issues are born scored.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
So new bugs are born scored like feature requests — issue_scores.py
lint would otherwise nag every freshly-filed bug.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@drsnuggles8, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 55 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 5b8d844a-9381-4458-b4e8-57595682b4d4

📥 Commits

Reviewing files that changed from the base of the PR and between 06f49ad and 59b87aa.

📒 Files selected for processing (5)
  • .github/ISSUE_TEMPLATE/bug_report.md
  • .github/ISSUE_TEMPLATE/feature_request.md
  • docs/README.md
  • docs/process/issue-scoring.md
  • scripts/issue_scores.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@drsnuggles8
drsnuggles8 merged commit 3a924fc into master Jun 29, 2026
6 of 8 checks passed
@drsnuggles8
drsnuggles8 deleted the feature/issue-scoring-system branch June 29, 2026 09:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant