Layer 3 of point-in-time support. Builds on #50 (as-of reads) and #51 (gc); makes the damage in #49 impossible rather than merely detectable.
Problem
DESIGN.md states the invariant plainly:
A node, once written, is never modified or deleted.
That is true of the code — nothing in src/ issues a DELETE against analysis_nodes — but it is a convention, not a property. Any hand-written SQL, any future maintenance script, any well-meaning cleanup can violate it, and the system has no way to notice or recover. That is not hypothetical: it happened, twice, and cost 67,468 nodes plus a broken evidence trail across 689 summaries.
The deeper issue is that deletion destroys the very thing point-in-time support depends on. An as-of read (#50) can only reconstruct history that is still present; anything deleted is absent from every view of the past, including views of moments when it was legitimately there.
Proposed change
Stop deleting. Retract instead:
ALTER TABLE analysis_nodes ADD COLUMN retracted_at TEXT; -- NULL = live
ALTER TABLE analysis_nodes ADD COLUMN retracted_by_run TEXT; -- provenance
What this buys
- The invariant becomes real. Append-only stops depending on everyone's good behaviour.
- Rollback becomes reversible. Undoing a bad
gc is clearing a column, not re-running analysis. Both of today's cleanups would have been reversible.
- History stays complete. Retraction is itself a dated event, so "what did the graph look like at T" stays answerable even for things later removed.
- It matches the stated non-goal. DESIGN.md already says "No eager deletion of superseded analysis — old versions are kept for comparison; reclaiming space, if ever needed, is a separate, deliberate act." Retraction is that separate deliberate act, made explicit.
Costs, honestly
- Every query must filter, or go through the view. Missing the filter somewhere yields subtly wrong results — the main risk of this change, and the reason the view should be the only sanctioned read path.
- Storage is not reclaimed. That is the point, but it needs a real
purge --retracted-before <ts> escape hatch for anyone who actually needs the space.
prospect verify should check retracted nodes too; a retracted node is still a node and its content must still hash correctly.
Layer 3 of point-in-time support. Builds on #50 (as-of reads) and #51 (gc); makes the damage in #49 impossible rather than merely detectable.
Problem
DESIGN.md states the invariant plainly:
That is true of the code — nothing in
src/issues aDELETEagainstanalysis_nodes— but it is a convention, not a property. Any hand-written SQL, any future maintenance script, any well-meaning cleanup can violate it, and the system has no way to notice or recover. That is not hypothetical: it happened, twice, and cost 67,468 nodes plus a broken evidence trail across 689 summaries.The deeper issue is that deletion destroys the very thing point-in-time support depends on. An as-of read (#50) can only reconstruct history that is still present; anything deleted is absent from every view of the past, including views of moments when it was legitimately there.
Proposed change
Stop deleting. Retract instead:
gc(prospect gc: a supported inverse for a run or an analyzer #51) setsretracted_atrather than issuingDELETElive_nodesview (WHERE retracted_at IS NULL)created_at <= T AND (retracted_at IS NULL OR retracted_at > T)— which correctly shows a node that existed at T and was retracted latermissingand is recomputed, exactly as todayWhat this buys
gcis clearing a column, not re-running analysis. Both of today's cleanups would have been reversible.Costs, honestly
purge --retracted-before <ts>escape hatch for anyone who actually needs the space.prospect verifyshould check retracted nodes too; a retracted node is still a node and its content must still hash correctly.