Summary
Add automatic quality scoring for agent outputs based on completeness, relevance, and format compliance, feeding scores back to reputation system.
Motivation
Currently, agent quality is only assessed by human review. Automatic scoring provides immediate quality feedback and enables data-driven agent selection.
Acceptance Criteria
Files to Modify
| File |
Action |
backend/src/services/qualityScorer.ts |
New: output quality scoring service |
backend/src/services/qualityScorer.types.ts |
New: scoring types |
backend/src/coordinator/coordinator.ts |
Add quality scoring after agent execution |
backend/src/agents/research/research.ts |
Define research output schema for scoring |
backend/src/agents/coding/coding.ts |
Define coding output schema for scoring |
backend/src/services/qualityScorer.test.ts |
New: scoring tests |
How to Contribute
- Create branch
feat/agent-quality-scoring
- Build quality scoring service with configurable rules
- Integrate scoring into coordinator workflow
- Define output schemas per agent type
- Write unit tests for scoring logic
- Run
npm run test && npm run lint before submitting PR
Summary
Add automatic quality scoring for agent outputs based on completeness, relevance, and format compliance, feeding scores back to reputation system.
Motivation
Currently, agent quality is only assessed by human review. Automatic scoring provides immediate quality feedback and enables data-driven agent selection.
Acceptance Criteria
Files to Modify
backend/src/services/qualityScorer.tsbackend/src/services/qualityScorer.types.tsbackend/src/coordinator/coordinator.tsbackend/src/agents/research/research.tsbackend/src/agents/coding/coding.tsbackend/src/services/qualityScorer.test.tsHow to Contribute
feat/agent-quality-scoringnpm run test && npm run lintbefore submitting PR