Skip to content

fix(skill-creator): scale verification to change risk - #282

Merged
daymade merged 1 commit into
mainfrom
fix/risk-scaled-skill-evals
Aug 8, 2026
Merged

fix(skill-creator): scale verification to change risk#282
daymade merged 1 commit into
mainfrom
fix/risk-scaled-skill-evals

Conversation

@daymade

@daymade daymade commented Aug 8, 2026

Copy link
Copy Markdown
Owner

Summary

  • add a pre-routing three-tier verification model
  • keep Tier 1 deterministic and Tier 2 limited to 1-2 with-skill samples
  • reserve paired baselines, grading, benchmark, and viewer for Tier 3
  • preserve specialized workflows, migration/sanitization/safety gates, and explicit user cancellation

Validation

  • quick_validate
  • security_scan
  • marketplace and documentation sync checks
  • changed-skill CI validator
  • 48-candidate old-bundle regression audit
  • fresh-context final review: CLEAN

No paired A/B eval, grader, benchmark, or viewer was run after the user declined that cost.

Route bounded fixes to targeted checks, narrow behavior changes to sampled replays, and reserve full paired evaluation for new, broad, high-risk, or explicitly benchmarked work.

Preserve migration and safety gates, specialized workflows, and the complete generic Tier 3 pipeline.

Co-Authored-By: Codex <noreply@openai.com>
@daymade
daymade merged commit 1c71e54 into main Aug 8, 2026
4 checks passed
@daymade
daymade deleted the fix/risk-scaled-skill-evals branch August 8, 2026 12:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant