fix(entity): stop validator over-rejecting real people, code tools, and event categories#178
Merged
Merged
Conversation
…nd event categories Production Gate-3 review of the entity-tag repair plan surfaced three over-rejection classes in the deployed validator (~1,600 of 7,677 planned removals were legitimate entities): - The _TOOL_OR_ORG_CONTEXT_HINTS branch rejected any person whose memory mentioned generic words like data/project/platform/tooling — in an engineering corpus that is nearly every memory, so 725 distinct multi-token person names were condemned. Person-shaped multi-token slugs now skip the context branch; camelCase and tool/org suffix signals still apply, and single-token brand-like people are still rejected contextually. - "code" as a primary markdown/code-fragment token rejected real tool names (claude-code, vs-code, code-server). It is now a secondary signal; people slugs containing "code" and path/markdown fragments remain rejected. - events and opportunities categories were missing from _CATEGORY_ALIASES, so every such tag was dropped as unknown_category. Backstop: a small common-word token set (bottom-line, deck-today, email-highlights, claude-desktop class) plus "pipeline" in the tech tokens keeps the generated-noise people slugs out now that the context branch no longer catches them. Empirical diff on a production clone (10,061 memories): rejections 7,677 -> 6,106; all 1,603 freed rejections are person/tool/event entities; all 32 newly-caught rejections are generated noise. Refs #72. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adjusts the entity-quality validator to reduce false rejections discovered during the production “Gate-3” review of the entity-tag repair rollout, specifically for real people names in technical context, tool names containing code, and missing events / opportunities categories.
Changes:
- Add
event(s)andopportunity/opportunitiesto the entity category alias map so they validate instead of being rejected asunknown_category. - Refine the “markdown/code fragment” heuristic by demoting
codeto a secondary token (so*-codetools aren’t rejected solely due tocode). - Reduce context-hint over-rejection for multi-token person names and add a backstop token set for common non-name word pairs; add tests covering these cases.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
automem/utils/entity_quality.py |
Updates category normalization and validation heuristics (people/tool/code/category handling). |
tests/test_entity_quality.py |
Adds regression tests for multi-token people-in-tech-context, code-suffixed tools, new categories, and common-word-pair noise. |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This was referenced Jun 10, 2026
jack-arturo
added a commit
that referenced
this pull request
Jun 10, 2026
…th (#179) ## Why PR #178's applied review suggestion (commit e6876b0, "Potential fix for pull request finding") gated the person-shape exemption on the display value containing a space: ```python if " " in (value or "").strip() and len(parts) >= 2 and _has_person_name_shape(parts): ``` Stored entity tags only retain the slug (`entity:people:jack-arturo`), so `validate_entity_tag(context=...)` — the path `scripts/lab/repair_entity_tags.py` uses — never satisfies the guard, and the context-hint branch re-rejects every real person mentioned alongside data/projects/tooling. **Empirical impact (prod dry-run, read-only):** 7,494 planned rejections with the guard vs ~6,106 expected with the exemption — ~1,390 legitimate person tags (jack-arturo ×51, zack-katz ×27, jason-coleman ×25, katie-keith ×14, ...) wrongly planned for removal. CI stayed green because every existing test exercised the spaced-value path (`validate_entity_value("people", "Mara Quinn", ...)`), never the slug path with context. ## What - Restore the plain person-shape exemption (no space guard) in `_looks_tool_or_org_like`. - Address the original Copilot concern (`entity:people:data-dog`) deterministically: add `"data"` to `_NON_PERSON_TECH_TOKENS`, so brand-like person-shaped pairs are rejected by the per-token vocabulary check on **every** path (with or without context) — strictly stronger than the context-hint rejection the guard tried to preserve. - Regression tests for the slug path: real-person tags survive technical context via `validate_entity_tag`; `data-dog` is rejected with `low_signal_people_slug` without needing context. ## Verification - `pytest tests/`: 490 passed, 12 skipped - Spot checks: `jack-arturo`/`zack-katz`/`jason-coleman`/`katie-keith` accepted with technical context on the tag path; `data-dog`, `growthmath`, `claude-code`-as-people still rejected Part of the issue #72 production repair rollout (follow-up to #178). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This was referenced Jun 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Production Gate-3 review of the entity-tag repair rollout (the human review of
rejected-tags.csvbefore executingrepair_entity_tags.pyagainst production) surfaced three over-rejection classes in the deployed validator. Of 7,677 planned tag removals, ~1,600 were legitimate entities.What was wrong
_looks_tool_or_org_likerejected any person whose memory content contained generic words (data,project,platform,tool...). In an engineering corpus that's nearly every memory — 725 distinct multi-token person names were condemned, including the corpus owner's own canonical entity.codeas a primary fragment token rejected real tool entities:claude-code(80 occurrences),vs-code.eventsandopportunitiescategories missing from_CATEGORY_ALIASES— every such tag was dropped asunknown_category.The fix
growthmath-style names are still rejected); single-token brand-like people (automem,claude) are still rejected contextually.codedemoted to_MARKDOWN_OR_CODE_SECONDARY_TOKENS(needs a second code-ish token to reject). People slugs containingcodeand path/markdown fragments remain rejected via existing checks.events/opportunities(+ singular aliases) added to the category map._NON_PERSON_COMMON_TOKENS(bottom-line / deck-today / email-highlights / claude-desktop class) andpipelineadded to_NON_PERSON_TECH_TOKENS.Empirical validation on a production clone (10,061 memories)
Every freed tag inspected by category: person names,
claude-code/vs-code,entity:events:*,entity:opportunities:*. Every newly-rejected tag is generated noise (good-plugins,chrome,claude-desktop,stream-deck-as-person).This also stops enrichment-time over-stripping: the same validator gates
_validated_entities(), so new memories were silently losing these entities on every store since #176 deployed.Test Plan
pytest tests/→ 487 passed, 12 skipped (env-dependent)make lint→ cleanRefs #72. Part of the staged production entity repair (Gate 3 of the rollout runbook).
🤖 Generated with Claude Code