fix(lexicon): a phrase must contribute beyond its component words

[elecnix]

Follow-up to #45, which introduced the regression this fixes.

What the full corpus run exposed

At scale, phrases were 84% of all adjudications (65,792 of 78,437 entries) and 75% of all hits (28,179 against the word lexicon's 9,350) — and they were overwhelmingly redundant:

do not     x563     is not    x475     does not  x268
with no    x247     context no x289    it still  x303
and stop   x485     👍 on     x285     with 👍   x285

Every one is an ordinary word sitting beside a word that already signals on its own. 👍 on is the clearest: 👍 is itself a praise term, so pairing it with whichever word happened to follow manufactures a second hit each time.

Flag rates tell the same story:

	judged	flagged	rate
words	12,645	478	3.78% (healthy — matches the reference model)
phrases	65,792	4,384	6.66% on 5x the entries

The failure was in validation as much as in code. Phrases were checked against a two-session slice containing almost no bigrams, so the noise never appeared. Adjacent words in running prose are simply not idioms — laisse tomber was real, but drowning in catch no and to dag.

The rule: contribution

A phrase earns its place only when it says something its component words do not. Enforced in two places:

• Deterministically, at match time — a phrase hit is dropped when one of its component words already fired on the same turn. This holds whichever model judged the phrase, and takes effect without re-adjudicating anything.
• In the prompt — a phrase must be a fixed expression whose meaning is not the sum of its parts and must carry something its words do not. The failing cases are named outright, with an explicit warning that almost every adjacent pair is neutral.

laisse tomber still fires: neither laisse nor tomber is a signal, so nothing suppresses it. That is exactly the discriminator we want, and it is pinned by test.

Versions

frustration-lexicon → 1.3 (prompt), turn-frustration → 1.2 (suppression). Existing word verdicts stay valid; only phrases need re-judging.

Test plan

• npm test — 444 passing. New tests/component/phrase-redundancy.test.ts covers both directions: idioms survive, redundant pairs are dropped, an idiom and an independent word signal both land in one turn, and two independent idioms both land.
• node --import tsx test/integration/test-commands.ts — 21 passing.
• npx tsc --noEmit clean.
• Purged the 65,792 junk phrase verdicts from the working corpus, kept the sound word verdicts, and restarted the full run on this branch.

🤖 Generated with Claude Code

[elecnix]

Closing unmerged. The suppression rule in this PR cut phrase noise substantially (flag rate 6.66% → 1.65%, hits 28,179 → 1,525), but the phrases that survive are still not idioms:

praise      | accepted or        frustration | after another
praise      | accepted feedback  frustration | again for
frustration | aach wbaagb        frustration | admin's mock

top hits:  first quality x111 · end your x69 · re check x43 · an explicit x26

Three attempts — original, prompt tightening, deterministic suppression — and the output is still wrong in kind rather than merely in volume. The premise was flawed: adjacent word pairs in running prose are overwhelmingly not idioms, and per-pair LLM adjudication cannot reliably find the rare real ones among tens of thousands of candidates.

Reverting the phrase feature instead. See #40, reopened with the measurements.