fix(memory-store): supersession + per-fact provenance end-to-end - #36
Merged
Conversation
The schema already supported soft-deprecation (`deprecated_at`, `deprecation_reason`, `superseded_by_summary_id` plus a partial `WHERE deprecated_at IS NULL` index), and `deprecateMessages` was implemented on `RawMessageStorageManager`, but the search layer, CLI, and synthesis prompt never saw them. A user repro (three versions of a decision, two deprecated) saw all three tie at similarity 1.000 and the lex tiebreaker picked arbitrarily; per-fact provenance (`metadata.source`) was visible in `list --json` but stripped from `search --json` and from the LLM synthesis context. There was also no CLI `deprecate` subcommand, so users had to drop into JS. Plumbs `includeDeprecated` through `SearchInput` → `UnifiedMemorySearchInput` → the ANN/lexical deps and the SQLite fallbacks. Adds `--include-deprecated` to `opencontext search` (default off, preserves current-truth). Adds a new `opencontext deprecate` subcommand that mirrors `add`'s shape and surfaces `--reason` / `--superseded-by`. Surfaces per-fact `metadata.source` through `SearchEvidence` → CLI `--context-only` output → the `buildSynthesisPrompt` line format. Wraps `searchChunksWithVectorTable` in a widen-and-retry loop so vec0 KNN preserves `limit` even when most top-K candidates are deprecated; mirrors that pattern on `searchMessagesWithVectorTable` (which already had a similar loop). Verified via the user's repro (REST → GraphQL → tRPC): top-1 search returns tRPC only, `--include-deprecated` returns all three, evidence envelope carries `metadata.source = "meeting://2026-08-15"`, and `deprecate` is idempotent. Existing tests pass (the only pre-existing failures on `main` — `applicability-propagation.test.ts:292` and `cli-shared.test.ts` rag resolution — are unrelated to this change). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Peefy
force-pushed
the
fix/supersession-and-provenance
branch
from
September 11, 2026 04:27
a464cb6 to
d70e54f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The schema already supported soft-deprecation (
deprecated_at,deprecation_reason,superseded_by_summary_idplus a partialWHERE deprecated_at IS NULLindex), anddeprecateMessageswas implemented onRawMessageStorageManager, but the search layer, CLI, and synthesis prompt never saw them. A user repro (three versions of a decision, two deprecated) saw all three tie at similarity 1.000 and the lex tiebreaker picked arbitrarily; per-fact provenance (metadata.source) was visible inlist --jsonbut stripped fromsearch --jsonand from the LLM synthesis context. There was also no CLIdeprecatesubcommand, so users had to drop into JS to deprecate anything.Changes
includeDeprecatedend-to-end — adds the field toSearchInput/UnifiedMemorySearchInput, threads it throughrunSemanticSearchForEmbeddingandrunLexicalSearchForKeywords(both the dep functions and the SQLite fallbacks), and adds--include-deprecatedtoopencontext search(default off, preserves current-truth).opencontext deprecateCLI subcommand — mirrorsadd's shape; flags--user(required),--id(repeatable, required),--reason,--superseded-by,--json,--help. CallsgetRawMessageManager().deprecateMessages(...).SearchEvidence.metadatanow carries the underlying hit's metadata. CLI--context-onlyprints asource: <uri-or-em-dash>line per evidence item.buildSynthesisPromptline format is now[n] (id, source=<uri-or-tier-or-'unknown'>, score=N.NNNN) snippet, withmetadata.sourcetaking precedence over the channel-levelsource.searchChunksWithVectorTablein a 3-attempt widen-and-retry loop (cap 4096 to respect sqlite-vec'sk value in knn query too largelimit) solimitis preserved even when most top-K candidates are deprecated. Adds a cross-reference comment on the parent-level mirror insearchMessagesWithVectorTableso the two vec0 KNN paths stay in sync.Verification
User's repro (clean SQLite-backed memory store):
Test plan
pnpm --filter @melandlabs/opencontext test— 109/109 pass (added 3 tests: defaultincludeDeprecatedis omitted,--include-deprecatedforwardstrue,--context-onlysurfaces per-factmetadata.source)pnpm --filter @melandlabs/memory-store test— 203/203 search tests passpnpm --filter @melandlabs/sqlite test— 23/23 passpnpm --filter @melandlabs/indexeddb test— 10/10 passadd× 3 →deprecate× 2 →search --limit 1returns tRPC only;--include-deprecatedreturns all 3; idempotent re-deprecatereports count=0The two pre-existing test failures on
main(applicability-propagation.test.ts:292readonly/mutable cast, andcli-shared.test.ts@melandlabs/rag/chroma-vector-storeresolution) are unrelated to this change — both confirmed pre-existing viagit stashround-trip.🤖 Generated with Claude Code