Summary
Audit of src/agentready/data/default-weights.yaml against all registered assessors found three discrepancies that cause silent scoring drift.
Issues
1. Typo: separation_concerns vs separation_of_concerns
The yaml has:
separation_concerns: 0.03
But the assessor (SeparationOfConcernsAssessor) returns attribute_id = "separation_of_concerns". The yaml key never matches any assessor, so it is dead weight. The assessor falls back to default_weight=0.03, which happens to be the right value, but the dead yaml key contributes 3% to the rescaling denominator without any assessor claiming it. This means every other yaml-registered attribute is very slightly overweighted.
Fix: Rename separation_concerns to separation_of_concerns in the yaml.
2. repomix_config not in yaml
RepomixConfigAssessor was recently registered in create_all_assessors() (see #400) but is not in default-weights.yaml. It falls back to default_weight=0.02 and contributes to the scoring denominator outside the yaml's rescaling logic.
Fix: Add repomix_config: 0.02 to the yaml and reduce other weights to keep the total at exactly 1.0.
3. branch_protection not in yaml
BranchProtectionAssessor is not in the yaml and uses default_weight=0.005. In practice this is benign today because the assessor always returns not_applicable (it requires GitHub API access and is skipped from scoring). But it should be documented.
Fix: Either add branch_protection: 0.005 to the yaml, or add a comment noting it is intentionally omitted as a conditional/API-dependent assessor.
How to reproduce
import yaml
from pathlib import Path
with open("src/agentready/data/default-weights.yaml") as f:
weights = {k: v for k, v in yaml.safe_load(f).items() if isinstance(v, float)}
# Dead key -- no assessor has this ID
assert "separation_concerns" in weights # passes (shouldn't)
assert "separation_of_concerns" not in weights # passes (wrong)
# Missing entries
assert "repomix_config" not in weights # passes (wrong)
assert "branch_protection" not in weights # passes (debatable)
Impact
- All assessments are affected by the
separation_concerns typo (slight overweighting of other attributes).
- Any assessment where
repomix_config is not skipped gets its weight applied outside the normalized yaml total.
Opened by Claude Code under the supervision of Bill Murdock.
Summary
Audit of
src/agentready/data/default-weights.yamlagainst all registered assessors found three discrepancies that cause silent scoring drift.Issues
1. Typo:
separation_concernsvsseparation_of_concernsThe yaml has:
But the assessor (
SeparationOfConcernsAssessor) returnsattribute_id = "separation_of_concerns". The yaml key never matches any assessor, so it is dead weight. The assessor falls back todefault_weight=0.03, which happens to be the right value, but the dead yaml key contributes 3% to the rescaling denominator without any assessor claiming it. This means every other yaml-registered attribute is very slightly overweighted.Fix: Rename
separation_concernstoseparation_of_concernsin the yaml.2.
repomix_confignot in yamlRepomixConfigAssessorwas recently registered increate_all_assessors()(see #400) but is not indefault-weights.yaml. It falls back todefault_weight=0.02and contributes to the scoring denominator outside the yaml's rescaling logic.Fix: Add
repomix_config: 0.02to the yaml and reduce other weights to keep the total at exactly 1.0.3.
branch_protectionnot in yamlBranchProtectionAssessoris not in the yaml and usesdefault_weight=0.005. In practice this is benign today because the assessor always returnsnot_applicable(it requires GitHub API access and is skipped from scoring). But it should be documented.Fix: Either add
branch_protection: 0.005to the yaml, or add a comment noting it is intentionally omitted as a conditional/API-dependent assessor.How to reproduce
Impact
separation_concernstypo (slight overweighting of other attributes).repomix_configis not skipped gets its weight applied outside the normalized yaml total.Opened by Claude Code under the supervision of Bill Murdock.