feat(#5904 PR-b): flow enrichment side-table — phantom-edge writeback stops rewriting graph.fb (#5915)#5923
Merged
Merged
Conversation
…PR-b slice 1) Add internal/graph/flows: a per-repo <stateDir>/flows.json side-table holding the cross-repo-aware flow DELTA (re-synthesised SCOPE.Process/SCOPE.EventFlow entities, their STEP/ENTRY/SEED edges, and the phantom cross_repo CALLS edges). The read-time merge is REPLACE, not additive: Apply() strips the index-baked intra-repo flows + their structural edges first, then substitutes the sidecar's cross-repo-aware set. MergeInto() is the shared Doc-path seam; absence / corruption / staleness fall back to the baked intra-repo flows (valid degraded state). source_key derives from CurrentGraphDescriptor (seg-aware, #5915 P2); atomic tmp+rename split Unix/Windows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#5904 PR-b slice 2) Merge the per-repo <stateDir>/flows.json flow side-table onto the graph at READ time with REPLACE semantics across all three consumer seams, mirroring PR-a's description overlay: - internal/mcp: applyFlowOverlay publishes the cross-repo flow DELTA into LoadedRepo.flowOverlay (per-repo memoized by sidecar+graph mtime, race-safe under readerMu). The forEach* iterators SUPPRESS baked SCOPE.Process/ SCOPE.EventFlow entities and INJECT the sidecar's (flowFilteredYield + injectFlowEntities, Doc and mmap paths); getStepAdj REPLACES the STEP_IN_PROCESS adjacency with the sidecar's; getCallsAdj ADDs the phantom cross_repo CALLS edges. Absence/stale -> baked intra flows (degraded-valid). - internal/dashboard: loadGroupForRef flows.MergeInto onto dr.Doc so the flow handlers (loop Doc.Entities/Relationships by Kind) see the merged set. - internal/cli/export: loadGroupGraphForExport merges per repo so the offline GraphML/Cypher/SVG/HTML export carries the cross-repo-aware flows. Tests DRIVE the real grafel_traces tool (list/get, mmap + Doc), the dashboard load seam, and the export command: cross-repo not doubled (exact count), an explicit additive double-count guard, single-repo parity, staleness fallback, and -race on concurrent reads vs sidecar re-publish. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…able (#5904 PR-b slice 3) runPhantomEdgePass keeps computing the cross-repo flow enrichment exactly as before (PromoteToPhantomEdges + RunProcessFlowWithCompanions + RunEventFlow + ApplyAsyncTriggerEdges), but changes only the SINK: instead of rewriting the whole graph (fbwriter.WriteGraphGen + graph.WriteAtomic over graph.fb/graph.json), it extracts the DELTA (the re-synthesised SCOPE.Process/SCOPE.EventFlow entities + their STEP/ENTRY/SEED edges + the phantom cross_repo CALLS edges) and writes it to the per-repo <stateDir>/flows.json side-table via flows.Upsert. graph.fb is NOT rewritten — for a #5901 segment-set this removes the #5915 P1 whole-graph collapse/OOM. The pass is the sole writer of the flow side-table, so it also clears a stale flows.json for any loaded repo that is no longer cross-repo-affected (a link removed without a reindex) — the read overlay must never resurrect old flows. The regression test now asserts the new contract: the flows land in flows.json (fresh, with the phantom CALLS edge) and graph.fb / graph.json are byte-identical (mtime unchanged — the no-graph-rewrite guard mirroring PR-a's TestWriteback_success). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…CE coverage Address the two review blockers on PR-b: 1. CORRECTNESS — total-link-removal leaked a stale sidecar. runPhantomEdgePass early-returned on an empty links file ABOVE the clear-stale-flows.json loop, so when the last cross-repo link was removed a repo's obsolete flows.json was never cleared and (its source_key still matching an un-reindexed graph) kept serving stale cross-repo flows forever. Drop the early return; with no links affectedRepos is empty and every loaded repo falls into the cleanup path. New test drives the pass with an EMPTY links set and asserts the sidecar is cleared (proven RED with the early return reinstated). 2. TEST COVERAGE — the SCOPE.EventFlow REPLACE path was unguarded (dropping both event-flow edge kinds from the strip set stayed green). Add TestApply_ReplaceSemantics_EventFlow (flows) + TestDashFlowOverlay_EventFlowReplace (dashboard seam): a baked SCOPE.EventFlow + its SEED_OF_EVENT_FLOW / STEP_IN_EVENT_FLOW edges must be suppressed (no dangling baked edge) and substituted by the sidecar's. Both proven RED when the event-flow kinds are dropped from isFlowStructuralEdge. Also (non-blocker): TestFlowOverlay_StepAdjReplace_Mmap asserts getStepAdj is REPLACE (not ADD) on the flag-ON mmap path — proven RED under an ADD mutation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Moves the cross-repo phantom-edge / flow-entity enrichment (the last #5915 graph-rewriting writeback) off
graph.fband onto a per-repo side-table (<stateDir>/flows.json), merged at read time with REPLACE semantics across all consumer seams. Companion to PR-a (descriptions). Option C (store the synthesized entities; cheap uniform merge) — chosen over recompute-at-read because the flows are read through three seams.Why REPLACE (not additive)
Flow entities have two producers: the normal index bakes INTRA-repo
SCOPE.Process/SCOPE.EventFlowintograph.fb; the phantom pass re-synthesizes cross-repo-AWARE ones. Sograph.fbkeeps the baked intra flows and the overlay must suppress them + substitute the sidecar's + add the phantomcross_repoCALLS — a naive additive merge would double every flow. Enforced at one point:flows.Apply=StripBakedFlows(removesSCOPE.Process/SCOPE.EventFlow+ENTRY_POINT_OF/STEP_IN_PROCESS/SEED_OF_EVENT_FLOW/STEP_IN_EVENT_FLOW) → substitute → add phantom CALLS.Three seams (all covered)
flowOverlayonLoadedRepo(memoized, underreaderMu);view_iter.gosuppress+inject on Doc AND mmap paths;getStepAdjREPLACE;getCallsAdjADD phantom CALLS.flows.MergeIntoat thegraphstate.goload point (after PR-a's description merge).loadGroupGraphForExportmerges per-repo; v2 zip shipsflows.json.Writeback
runPhantomEdgePasscomputation unchanged; sink swapped fromWriteGraphGen/WriteAtomictoextractFlowDelta+flows.Upsert.graph.fbnever rewritten. The pass (sole writer) clears stale sidecars for no-longer-affected repos, including the total-link-removal case.Verification (independent adversarial review — APPROVE, incl. a REQUEST-CHANGES round)
grafel_tracestool).functions) → RED; strip set matches the synthesizer's emitted kinds exactly.WriteAtomicreinsert → RED); composition with PR-a descOverlay;-raceclean.SCOPE.EventFlowREPLACE coverage added (drop event-flow strip kind → RED with dangling baked edges); getStepAdj REPLACE mmap test.Segment-set mmap serving is Doc-path-only until #5912 (noted).
Refs #5904
Refs #5915
🤖 Generated with Claude Code