Skip to content

Add aggregate AI structure signals - #6

Draft
theclaymethod wants to merge 1 commit into
mainfrom
agent/add-ai-structure-signals
Draft

theclaymethod wants to merge 1 commit into
mainfrom
agent/add-ai-structure-signals

Conversation

@theclaymethod

Copy link
Copy Markdown
Owner

What changed

  • Detect repeated rhetorical triads as an aggregate structure signal.
  • Flag prose where long sentences repeatedly chain clauses with “and”.
  • Flag long prose with unusually sparse internal punctuation.
  • Exempt reference documentation that legitimately uses factual triads, ordered technical operations, or sparse declarative prose.
  • Add six adversarial red/green evals and update the structure detector guidance.
  • Precompile literal phrase regexes and cache line contexts so the large-document performance gate remains below its limit.

Why

The Economist’s 2026 corpus comparison found that current LLM prose is better identified by aggregate vocabulary, punctuation, and sentence-structure habits than by isolated words or em-dash use. UNSLOP already covered negative parallelism, long average sentences, uniform cadence, academic excess vocabulary, and em-dash clusters, but its triad metric was not enforced and it did not detect conjunction-heavy long sentences or punctuation-sparse long prose.

This change adds those generalizable structure signals without banning context-dependent words such as “significant”, “parameter”, or “consequences”.

Validation

  • python3 evals/run_adversarial.py
  • python3 evals/build_shared_benchmark.py --check
  • python3 evals/check_taboo_parity.py
  • python3 evals/check_pattern_coverage.py
  • python3 evals/kata_add_pattern.py --run
  • python3 scripts/check_packs.py
  • skill-benchmark validate evals/shared-benchmark.json --strict-leakage
  • git diff --check

All deterministic gates pass: 486 passes, the single existing documented XFAIL, and zero regressions. The optional behavioral tune matrix could not run because this environment’s Claude Code subscription access is disabled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant