Skip to content

Release v6.4.1: Fix syntactic dimension scoring bug - #45

Merged
drbothen merged 6 commits into
mainfrom
develop
Dec 15, 2025
Merged

drbothen merged 6 commits into
mainfrom
develop

Conversation

@drbothen

Copy link
Copy Markdown
Member

Summary

  • fix(dimensions): Use monotonic scoring for syntactic repetition instead of Gaussian
    • Documents with very low repetition (e.g., 0.06) were incorrectly scored as POOR because Gaussian scoring penalized values "too far" from the target
    • Research confirms: Human text ~38% template repetition, AI text ~95%
    • Lower repetition = more human-like, so monotonic scoring is correct
  • docs: Add Configuration Over Code story and architecture
  • chore: Bump version to 6.4.1

Test Results

  • temp2.md: Score improved 79.5 → 80.8, Syntactic POOR → GOOD
  • temp1.md: Score improved 81.1 → 82.4

Test plan

  • Verified scoring improvement on test documents
  • No breaking changes to scoring interface

- Add Story 8.1: Configuration Over Code implementation plan
- Add architecture doc 18-configuration-system.md with ConfigRegistry design
- Update component architecture diagram with Configuration System layer
- Add design principle 15.4: Configuration Over Code
- Update Epic 3 stories (3.1, 3.2, 3.3) to depend on Story 8.1
- Add configuration-system.md to devLoadAlwaysFiles
The Gaussian scoring incorrectly penalized documents with very low
syntactic repetition (e.g., 0.06 scored only 9/100). Research shows
AI text has ~95% template repetition vs ~38% for humans, so lower
repetition should always score higher.

Changed from Gaussian (target=0.27) to monotonic scoring where
lower repetition = higher score, using thresholds of 0.30/0.70.
@github-actions github-actions Bot added type: docs Documentation (maps to docs: commit) type: build Build/CI changes (maps to build: commit) area: dimensions Dimension analyzers labels Dec 14, 2025
@github-actions github-actions Bot added the type: test Test changes (maps to test: commit) label Dec 14, 2025
- Update syntactic tests for monotonic scoring (S-curve: 75 at threshold_low, 25 at threshold_high)
- Update dimension count tests (16 → 17 dimensions)
- Update weight assertions across all dimension tests
- Add missing dimension fixtures to high_quality_results (pragmatic_markers, figurative_language, semantic_coherence, energy, ai_vocabulary)
- Fix EnergyAnalyzer alias test to handle module reload issues
- Update dimension registration tests with new weights (perplexity: 3.0, burstiness: 8.0)
- Update weight distribution tests for 17 dimensions (add energy)
- Regenerate syntactic baseline scores for monotonic scoring (all 75.0)
- Update tier categorization test (SUPPORTING: 4 → 5)
@drbothen
drbothen merged commit d65a98e into main Dec 15, 2025
22 checks passed

This branch was previously deployed

1 inactive deployment
github-pages — 377c9312 Deployed Dec 15, 2025 by drbothen via deploy #36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: dimensions Dimension analyzers type: build Build/CI changes (maps to build: commit) type: docs Documentation (maps to docs: commit) type: test Test changes (maps to test: commit)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant