revert(lexicon): remove multi-word phrase support - #48
Merged
Conversation
Phrases do not work, and three attempts did not change that. Over the real corpus they were 84% of all adjudications (65,792 of 78,437 entries) and 75% of all hits (28,179 against the word lexicon's 9,350), and the output was noise. Tightening the prompt and then adding a deterministic rule that suppressed a phrase whose component word had already fired cut the volume hard — flag rate 6.66% → 1.65%, hits 28,179 → 1,525 — but what survived was still not idiomatic: praise | accepted or frustration | after another praise | accepted feedback frustration | again for frustration | aach wbaagb frustration | admin's mock top hits: first quality x111 · end your x69 · re check x43 · an explicit x26 The premise was wrong rather than the tuning. Adjacent word pairs in running prose are overwhelmingly not fixed expressions, and no judge reliably finds the rare real ones among tens of thousands of candidates. Reducing the noise floor does not turn a bad candidate set into a good one. What remains is what measurably works: single-word judgement (3.6% flag rate, producing `stuck`, `pedantically`, `salty`, `liar`, `masterful`, 🍾) and the lexicon-free paralinguistic markers, which need no model at all. `laisse tomber` and `trop lent` are genuinely lost. They are real, and #40 is reopened for them — but a future attempt needs a different mechanism, such as a curated idiom seed list or statistical n-gram selection, not per-pair LLM adjudication of every adjacent word pair. DESIGN.md records the withdrawal and why, so the next attempt starts from the measurements rather than from the same assumption. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reverts the phrase half of #45. Reopens #40.
Why
Phrases do not work, and three attempts did not change that.
Over the real corpus they were 84% of all adjudications (65,792 of 78,437 entries) and 75% of all hits (28,179 against the word lexicon's 9,350), and the output was noise.
Tightening the prompt, then adding a deterministic rule suppressing any phrase whose component word had already fired, cut the volume hard — flag rate 6.66% → 1.65%, hits 28,179 → 1,525. But what survived was still not idiomatic:
The premise was wrong, not the tuning. Adjacent word pairs in running prose are overwhelmingly not fixed expressions, and no judge reliably finds the rare real ones among tens of thousands of candidates. Lowering the noise floor does not turn a bad candidate set into a good one.
What remains
The parts that measurably work:
stuck,pedantically,salty,liar,hell,masterful,🍾???, elongation; no model requiredWhat is lost
laisse tomberandtrop lent— the multilingual idioms that motivated #40. They are real, and #40 is reopened. A future attempt needs a different mechanism: a curated idiom seed list, or statistical n-gram selection, rather than per-pair LLM adjudication of every adjacent word pair. DESIGN.md records the withdrawal and the measurements so the next attempt does not restart from the same assumption.Test plan
npm test— 424 passing (phrase-specific tests removed).node --import tsx test/integration/test-commands.ts— 21 passing.npx tsc --noEmitclean.🤖 Generated with Claude Code