The fact store (informally, "New Brain") is the memory organ: a store that holds what is known, not merely what was said. Every atom it keeps carries a verdict (is this believed true?), provenance (where did it come from, verbatim?), and a lifecycle (is it still current, superseded, or contested?). That triad — verdict + provenance + lifecycle — is the whole difference between a fact store and a note pile.
This document is a rebuild-grade blueprint. An outside AI should be able to stand the system up on its own Postgres + pgvector from what is written here: the schema, the seven fact kinds, the write path, the conflict pipeline, the read/recall labeling, and the tag registry governance. It also documents, honestly, where the system does not yet work.
Everything reaches the database through one seam. A single process owns the connection
and exposes a small set of tools (describe, remember, recall, resolve_conflict,
resolve_both_stand, open_conflicts, registry_query, registry_add_leaf,
registry_propose_root). No other code queries the store directly, ever. Rebuild that
discipline first — it is what keeps the lifecycle rules enforceable in exactly one place.
The store runs on Postgres with the vector extension (pgvector). Statement embeddings
are 768-dimensional, produced by a local embedding model (nomic-embed-text over an Ollama
HTTP endpoint). The conflict-density check and recall ranking both ride an HNSW cosine
index over those embeddings — indexed SQL, never in-process vector math.
The load-bearing law: every field that retrieval or lifecycle logic reads is a
first-class column. A JSON meta column exists only for the non-load-bearing long
tail (raw topic lists, dropped-topic audit trails, absorbed-clause provenance). Nothing
that recall or the conflict pipeline consults ever lives inside meta. This is a
deliberate correction of an earlier "everything is a blob" design that made lifecycle
logic impossible to enforce.
facts — the typed, verdict-carrying atoms. Common frame plus per-kind columns:
- Common frame:
id,statement(one atomic assertion, paraphrased),kind(one of the seven),tags(canonical dotted tags, atext[]),verdict. verdictis one of:active,superseded,withdrawn,parked,contested.- Evidence pointer:
evidence_quote(verbatim span from the source),evidence_source_type(utterance|capture|file|commit|url),evidence_source_ref(session+timestamp, capture id,path:line, commit sha, or url),evidence_captured_by(the scribe model — provenance of the writer, not the subject). - Temporal frame:
t_said(when it was uttered),t_captured,t_last_verified,t_superseded. - Supersession links:
supersedes,superseded_by(self-referential FKs). - Per-kind load-bearing columns (nullable, populated only by the kind that owns them):
scope,status_detail,trigger_predicate,risk_weight,earned_by,subject,expected_lifetime. embedding vector(768), andmeta jsonbfor the long tail.
Indexes: an HNSW cosine index on embedding (drives both conflict density and recall),
a GIN index on tags, and btree indexes on kind and verdict.
conflicts — contested pairs and their resolution reasoning. Columns: fact_a,
fact_b, status (open | resolved), winner (set on resolution; NULL marks a
both-stand close), reasoning, created_at, resolved_at.
tags — the machine-owned registry: canonical dotted paths (tag PK), a self-FK
parent (NULL means a root), a description, and created_at.
tag_aliases — free-text spellings (alias PK) mapping to a canonical tag. This is
where fragmentation heals: many spellings, one canonical tag.
documents — the raw-capture layer for anything not yet a typed fact. content,
source_ref, embedding, meta. Recall falls through here (labeled as unverified
document content) when the fact layer is thin.
Every fact is exactly one kind. The kind determines which per-kind columns are load-bearing and how the lifecycle treats it. All examples below are synthetic — invented to illustrate shape, not drawn from any real record.
Owns: scope (what it governs), status_detail (locked | parked-with-trigger),
trigger_predicate (for a parked decision, the event that would reopen it).
statement: The build pipeline ships only through the staging gate; no direct pushes to production. kind: DECISION · scope: build pipeline · status_detail: locked evidence_quote: "everything goes through staging first, no exceptions, that's final"
Owns: earned_by (the evidence or experiment that established it).
statement: Always write a failing test that reproduces a bug before fixing it. kind: RULE · earned_by: "the March regression that shipped because no test pinned the old behavior" evidence_quote: "from now on, repro test first, then the fix — no exceptions"
Owns: subject (the thing whose state this is), expected_lifetime (re-verify cadence).
statement: The nightly integration job is currently green on the
sample-svcbox. kind: STATE · subject: sample-svc · expected_lifetime: 2d evidence_quote: "sample-svc is green right now"
STATE is the only kind with staleness. When it ages past its lifetime it stops serving as live truth (see §5).
No per-kind columns. Identity, background, tastes, hardware — things that do not perish.
statement: The user prefers dark-mode editors and dislikes light themes. kind: PREFERENCE evidence_quote: "I can't stand light-mode editors"
No per-kind columns. A path, URL, repo, dashboard, or doc.
statement: The service runner token is stored at C:\ops\sample\runner.env. kind: REFERENCE evidence_quote: "the runner token lives at C:\ops\sample\runner.env"
REFERENCE facts that name a filesystem path can be resolved mechanically in the write path (see §4, lane 2).
Owns: risk_weight (probability × blast radius, as assessed), trigger_predicate (what
would reopen it). A RESIDUAL is a risk-accepted gap — it ships on purpose, as a known
issue, distinct from an unassessed gap.
statement: The importer does not validate UTF-16 input; accepted because all current feeds are UTF-8. kind: RESIDUAL · risk_weight: "low probability × contained blast radius" trigger_predicate: "a non-UTF-8 feed is onboarded" evidence_quote: "we'll live with the UTF-16 gap until a non-UTF-8 feed shows up"
No per-kind columns. A milestone or history. An EVENT makes no claim about the present; a completed action's outcomes (counts, scores, check results) are absorbed into the event, not emitted as standalone facts.
statement: A doc migration moved all 214 pages; checksum parity verified (214 == 214) and a link sweep found zero broken links. kind: EVENT evidence_quote: "migrated all 214 pages. Checksum parity verified (214 == 214), link sweep found zero broken links"
Every fact carries a verbatim quote. The extractor guarantees it: the model emits a quote
field, and anchor hygiene in code drops any fact whose quote does not actually appear
in the source (whitespace-normalized, backslash-run-collapsed). A fact that cannot be traced
to its input is not written — the same discipline as a code extractor refusing a symbol it
cannot find in the file.
The four evidence columns capture the full chain: what was said (evidence_quote), how it
reached the store (evidence_source_type), where to find the original (evidence_source_ref),
and which model wrote it (evidence_captured_by). That last one is provenance of the
writer, not aboutness — the scribe model's name is never a topic. A capture written by an
assistant is not about that assistant.
remember(text) runs one capture through the learning loop:
input → extract (typed personal schema) → tag (translate free-text → registry)
→ SYNCHRONOUS conflict-density check (pgvector kNN) → triage → write
A local extraction model reads the capture and emits typed fact records under a one-shot prompt. Three disciplines carry over from the code-extraction lineage: a single worked example, anchor hygiene (§3), and a precision rule — extract only what the text explicitly asserts, never an inferred scope, owner, risk, or value. Untypeable, vague, or purely narrative content yields nothing and stays a document.
Two deterministic passes run in code because the small extraction model cannot reliably hold the structure in-prompt:
- Index-page gate: a link-dense capture (many
[[wiki-links]]) is a table of contents; its assertions are second-hand restatements. It routes wholesale to the documents layer rather than minting mis-attributed facts. - Outcome absorption: an adjacent, outcome-shaped EVENT ("verified", "N of N matched",
"scored Y") is folded into the preceding EVENT it reports on. Absorbed statements and
quotes are preserved under
meta.absorbed_outcomes— a wrong absorption degrades presentation, never truth or traceability.
The person never types a tag. They speak naturally; the write path translates free-text
topics into canonical registry tags (§6). Before translation, topic hygiene runs
deterministically: scribe/model names are dropped (provenance-is-not-aboutness), topics are
deduped case-insensitively, and dropped topics are recorded in meta — never silently lost.
Unresolved topics that are entity-shaped (an uppercase letter, a digit, or a dot path)
surface to the person as candidate roots; lowercase common-word noise stays in meta only.
A capture that yields no typed facts lands in the documents layer (its only home). A capture that does yield facts also lands whole in the documents layer, exactly once. This is deliberate: anchor hygiene can drop a true-but-unquotable detail from the facts, so keeping the raw capture makes recall failures re-minable later. Live captures get the same minable safety net that migrated content does.
Before a fact is committed, its statement embedding is compared (pgvector HNSW kNN, optionally restricted to shared tags) against nearby active or contested facts. A near-hit of the same kind above the similarity threshold is a conflict candidate — unless it is an exact normalized restatement of the same fact, which is a pure re-capture, not a conflict.
Known imprecision (a documented gap): the boundary between "a near-paraphrase of the same fact, reworded" and "the same fact with a changed value" is not measured. A reworded restatement can trip a false interrupt. This is the safe failure mode — it asks rather than silently merging — but its precision is unmeasured and tuning is future work. It is filed as an open question (§7).
On a genuine conflict candidate, the write path routes to exactly one lane:
- RULE exception (checked first): a RULE-kind conflict never auto-resolves. A rule replacing a rule always runs the resolution round, explicit supersession or not. Both facts go contested and an interrupt is raised.
- Lane 1 — explicit supersession: the capture explicitly says it replaces a prior belief. Auto-resolve: write the new fact, supersede the old, link them.
- Lane 2 — mechanically verifiable: for STATE/REFERENCE facts naming a filesystem path, ground truth decides — the fact whose path actually exists wins, the other is superseded. (URLs are treated as un-checkable in the write path — no network — and fall through.)
- Lane 3 — genuinely contested: no explicit supersession, not mechanically resolvable. Both facts are written contested and a conflict interrupt is raised so the person can resolve it while context is warm.
The interrupt surfaces a head-to-head: both statements, both kinds, both evidence quotes,
both capture times, and the exact resolve_conflict(...) call to close it.
A conflict is a row in conflicts linking two facts. Opening a conflict flips both
facts to contested. It stays open until a human head-to-head call closes it. There are
two closing lanes, and the distinction is load-bearing.
resolve_conflict(conflict_id, winner_id, reasoning) closes a genuine either/or:
- The winner returns to
active. - The loser is superseded —
verdict = superseded,superseded_byset,t_supersededstamped, and the resolutionreasoningattached to the loser's supersession record. Supersession is structural, not prose: the link and timestamp are what recall reads. - The conflict row records
status = resolvedand thewinner.
resolve_both_stand(conflict_id, reasoning) closes a pair as a non-conflict: the
density check false-fired on complementary facts that are both true and do not actually
contradict. Both facts return to active; neither is superseded. In the audit trail
a both-stand close is distinguished from a head-to-head by winner IS NULL.
This lane exists because without it the pipeline was forced to supersede a still-true fact every time the extractor over-split one assertion into two near-identical fragments. It refuses to run on an already head-to-head-resolved conflict — re-activating both facts there would resurrect a legitimately superseded loser.
The both-stand lane is the pressure valve for a real defect: sentence-shredding. The extraction model can split a single assertion — one inventory sentence, one compound statement — into two or more fragment facts. Those fragments are near-identical, so the density check fires a conflict between pieces of the same original claim. That is a false positive: there is no contradiction, only over-splitting. Both-stand is how such a pair is closed without falsely superseding a true fragment.
Status of the sentence-shredding defect: RESOLVED, pending merge. The granularity fix lives on a feature branch (
task/17) and is not yet merged. Receipt: an eval run moved the shred rate from 0.50 → 0.10. Treat it as resolved-pending-merge — fixed on the branch, not yet in the mainline. Until the merge lands, both-stand remains the operational valve.
Two other extractor incidents are not resolved and must not be conflated with the shred fix — see the first entry of §7.
recall(query) ranks facts by cosine similarity to the query embedding and returns each one
with an honest serving label. The store never launders a labeled fact into present-tense
truth; the caller sees the verdict, the age, the verified-since date, and any contested
partner, and honors them.
Only active, contested, and parked facts are retrievable; superseded and withdrawn
are never served. Relevance gates: below a floor similarity a fact is too weak to serve at
all; if the best fact is below a "strong" threshold the fact layer is thin and recall
falls through to the documents layer.
The labeling rules — this is the trinary discipline made into serving rules:
- active — believed current; serves as present-tense truth.
- expired STATE → "last known". A STATE aged past its
expected_lifetime(per-fact hint, else a default cadence) downgrades tostale-state: "last known as of<date>, unverified since — do not serve as live." It is not present-tense truth. - EVENT → never present-tense. History is not currency. An EVENT serves as "records that something happened; makes no claim about the present state."
- contested → both sides served, labeled, no silent pick. A contested fact serves alongside its contested partner; the label says the head-to-head is unresolved. The store refuses to choose for the reader.
- parked — served but flagged "set aside — not an active claim."
- thin fact layer → documents fall-through, labeled "unverified document content —
the fact layer had no strong answer; this is raw capture, not a typed fact." The response
says which layer answered (
fact,fact-weak,document, ornone).
The store keeps a triage queue of its own gaps — things assessed as not yet known rather than silently resolved either way. The first entry is a meta-entry: a gap the system holds about its own extractor.
1. Extractor negation-inversion and identity-mangle — cause unconfirmed (holding aporia).
Two extraction incidents were observed twice in production: a negation inversion (a
fact recorded with its polarity flipped — the "fact 74" incident) and an identity mangle
(a fact attached to the wrong subject — the "fact 161" incident). Neither has been
reproduced under synthetic eval, and the cause is unconfirmed. This is not a claim
that the extractor is defect-free, and it is not a claim that these are fixed — it is an
open, unresolved gap held honestly. (It must not be conflated with the sentence-shredding
defect of §5, which is resolved-pending-merge on task/17. These are separate.)
2. Conflict-density paraphrase precision — unmeasured (§4). The near-hit boundary between a reworded restatement of one fact and a genuine changed-value conflict is not measured. The current behavior errs safe (interrupt rather than silent-merge), but the threshold is untuned. Tuning is future work and needs a fresh eval set.
3. Per-tag STATE staleness defaults — not implemented. STATE staleness currently uses a
per-fact expected_lifetime hint or a single global default. Per-tag default cadences (so a
whole class of STATE ages on its own clock) are noted but not built.
The queue's discipline: a gap left open is only a failure when the risk of leaving it open outweighs the cost of closing it. High-risk-and-cheap gaps get resolved; high-risk-and- expensive gaps become blockers; low-risk gaps are legitimately parked and ship as known issues. An unassessed gap masquerading as "fine" is the dangerous case — it is entered here so it gets weighed rather than banked.
Tags are machine-owned and shaped as a dotted hierarchy (a gameplay-tag-style tree). The person never types a tag; the write path translates natural language into canonical tags. The registry has three governance rules:
- Roots are gated. A new root (a tag with no dotted parent,
parent IS NULL) is a human decision. The write path never invents a root — it surfaces an entity-shaped unresolved topic as a candidate and waits for approval (registry_propose_root). - Leaves auto-mint. A dotted input whose parent already exists mints its leaf
automatically, both inside tag resolution and via the explicit
registry_add_leaftool. Minting refuses a bare/rootless tag (that would be a back-door root) or a tag whose parent does not exist (that would be an orphan). - Aliases heal fragmentation. Free-text spellings resolve to a canonical tag through the alias table, so many phrasings collapse to one tag rather than splintering the tree.
Tag resolution order, given a free-text topic: alias match → exact tag match → unique leaf-name match (the last dotted segment is unique across the tree) → dotted-with-existing- parent auto-mint → otherwise unresolved (surfaced as a candidate root, never silently created).
- Provision Postgres with the
vector(pgvector) extension. Stand up a local embedding endpoint producing 768-d vectors. - Apply the schema:
tags,tag_aliases,facts,conflicts,documents, plus the HNSW, GIN, and btree indexes. Keep every load-bearing field a first-class column; reservemetafor the non-load-bearing tail only. - Route all database access through one seam process. Nothing else touches the DB.
- Implement the write path: extract → tag-translate → synchronous density check → triage (RULE-exception, then lanes 1/2/3) → write with verdict + evidence + temporal frame. Keep the lossless documents floor.
- Implement recall with the trinary labels: active current, expired STATE as "last known", EVENT never present-tense, contested serves both sides, thin-layer documents fall-through.
- Implement the conflict pipeline with both closing lanes: winner/loser supersession and
the
winner IS NULLboth-stand close. - Enforce registry governance: machine-owned tags, human-gated roots, auto-minted leaves, alias healing.
- Carry the open-questions queue forward — including the meta-entry about the extractor — so a rebuild inherits the system's honest account of what it does not yet know.