feat(#5904): description side-table — enrichment writeback stops rewriting graph.fb (#5915 P1)#5918
Merged
Merged
Conversation
… slice 1) Add internal/graph/descriptions: a per-repo <stateDir>/descriptions.json side-table (version, computed_at, source_key, results map[id]string) with an atomic tmp+rename write (Windows-safe split file), absence/corrupt/stale- tolerant Read, and an Upsert read-modify-write. The source-freshness key is derived from graph.CurrentGraphDescriptor (segment-set -> gen-dir/manifest mtime; single-file -> .fb mtime; JSON-only -> graph.json mtime), never the collapsed single-file path (5915 P2 discipline). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017quGgaqK7NRoxGT6BTqV2o
…slice 2)
Merge the per-repo <stateDir>/descriptions.json side-table onto entities at
READ time, mirroring the group-algo overlay:
- internal/mcp: applyDescriptionOverlay stamps PropSet("description") onto
lr.Doc.Entities (flag-OFF Doc path) and builds a LabelIndex.descOverlay
index-keyed table for the flag-ON mmap path (materializeFromReader /
materializeEntityOverlay additively PropSet it). Per-repo memoized by the
sidecar mtime + the repo reparse mtime; applied on reload and re-checked on
the Group() serving path. Absence/corrupt/stale -> no-op; a miss never clears
an entity's baked-in/extractor-native description (ADDITIVE guarantee).
- internal/dashboard: loadGroupForRef merges the sidecar onto DashRepo.Doc so
the PropGet("description") consumers (openapi/DAG export) surface write-back
descriptions now that write-back no longer bakes them into the graph.
Tests cover single-file (Doc + mmap paths), segment-set (Doc path; the MCP
serve path serves a segment-set from the collapsed Doc, Reader nil), the
additive no-clobber guarantee, and stale-after-reindex no-op.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017quGgaqK7NRoxGT6BTqV2o
…ide-table (#5904 PR-a slice 3) handleEnrichmentWriteback Target 1 no longer rewrites the whole graph (fbwriter.WriteGraphGen + graph.WriteAtomic on graph.json). It now upserts the description into <stateDir>/descriptions.json (source key from the graph descriptor) and invalidates the group cache so the next load re-merges the sidecar. The in-memory PropSet is kept for immediate visibility of the resident group; Target 2 (markdown doc) and Target 3 (audit) are unchanged. This removes the #5915 P1 hazard: for a #5901 segment-set the resident Document is a COLLAPSED union of every segment, so re-serialising it as one flat graph.<gen>.fb re-materialised the whole graph in memory (OOM) and discarded the segmented layout. The side-table leaves the graph file / generation untouched. The success test now asserts the description lands in descriptions.json and that graph.json is NOT rewritten with it (no collapse). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017quGgaqK7NRoxGT6BTqV2o
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Moves entity description enrichment off the graph rewrite and onto a per-repo side-table (
<stateDir>/descriptions.json), merged back at read time — the first #5904(e) slice and the #5915 P1 fix (enrichment must not rewritegraph.fb, which under the segmented format would force a whole-graph collapse).Slices
internal/graph/descriptions— per-repo side-table{version, computed_at, source_key, results: map[entityID]string}, atomic tmp+rename (split Unix/Windows),Read→(nil,false) on absent/corrupt/stale (source-key vsCurrentGraphDescriptor),Upsert.applyDescriptionOverlaymerges the sidecar in both the MCP mmap path (LabelIndex.descOverlayinmaterializeFromReader) and the Doc path, additively (never clobbers a baked-in description), memoized by sidecar mtime under the existingreaderMu/readRetiredsafety.handleEnrichmentWritebackTarget 1 now callsdescriptions.Upsert+ invalidate instead ofWriteGraphGen+WriteAtomic(no graph rewrite). Markdown + audit targets unchanged.Verification (independent adversarial review — APPROVE)
TestDescOverlay_Absent_NoClobber+ mmap variant both RED.WriteAtomic→TestWriteback_successRED; segment-set collapse path is gone.-race ×10clean, full mcp/dashboard/descriptions suites green.Segment-set mmap serving deferred to #5912 (Doc-stamp is the operative merge there for now).
Refs #5904
Refs #5915
🤖 Generated with Claude Code