MEMEX stores knowledge in Markdown Books and reasoning in trails between them.
The files under brain/ are authoritative. Search vectors, rendered views, and
routing indexes are derived. This document describes the current implementation;
CONNECTORS.md describes work that is not implemented.
The live agent selects records using memex/criterion.py. Each automatic record
requires a T1–T6 trigger, an inspectable referent, and a future question. Progress,
plans, unsupported conclusions, and facts cheaply recoverable from git or command
help are excluded. An expected rate is a diagnostic, never a quota.
memex note and memex new route through the guarded writer in edits.py.
recording.py validates admission metadata and wraps records with the actor,
mode, reference, and timestamp. The modes are:
| Mode | Meaning |
|---|---|
recorded |
Automatically selected under the criterion |
manual |
Explicit user request to retain information |
agreed |
Declared exact human assent |
Metadata makes a claim's basis inspectable. It does not verify the evidence, prove truth, or cryptographically authenticate assent. An agent with direct filesystem access can bypass the CLI. Apply the access policy.
Automatic replacement requires T6 and the exact old text. The previous record is preserved below the line. Research Books refuse replacement; new results append with dates. Records reject embedded internal markers and injected trail sections. Concurrent writers serialize with file locks and atomic replacement; stable record IDs and exact-text checks make retries idempotent.
A [[shelf/book]] link declares relatedness. A trail is a separate file with an
ordered ## Route; each step names its move, date, reason, and destination Books.
A Book can be reached by several trails. Its ## Trails through this Book section
is a derived list of those routes, refreshed by memex trail --sync.
Creating or changing a trail requires approval of the exact proposed route or
step. This is the Button. The CLI requires --agreed for these writes; Scribe
cannot perform them. Reasons must come from the record. Daily proposals quote
existing prose and decline links for which no adequate reason is present.
Trail coverage is descriptive, not a quota requiring every Book to join a route.
memex daily refreshes the index and queues supported trail proposals. It calls
no generative model, but can call local Ollama for embeddings. It installs no
schedule. Default pending and Stop nudges show trail proposals. pending --all
also exposes legacy fact candidates for agent review. Queue acceptance writes
before acknowledgment so a failed write does not silently discard a candidate.
Recall reads current Books and combines two rankings with reciprocal-rank fusion:
lexical matches, including exact phrases and facets, and semantic similarity from
local Ollama embeddings (bge-m3 at 127.0.0.1:11434). Provenance and archived
status affect ranking. One best passage per Book is returned, with its line number.
Chunks retain their provenance layer. Longer claims have separate chunks; very long text is split. Internal write metadata is removed from displayed excerpts. Facet text helps retrieval without becoming the returned passage.
state/recall-index.json is a cache. Its version, model identity, and content
hashes control rebuilding. Changes to a Book's title, body, facets, or privacy
label invalidate it. Missing vectors are retried on later reindexing. Index
writes are locked and atomic.
Recall also reads current content and labels from disk: new or changed Books
remain lexically searchable before reindexing, deleted Books disappear, and
--no-vault follows the current label. When embeddings are unavailable, the
fallback is reported explicitly. --full expands excerpts; --json exposes
structured results. Always read the source before citing an excerpt.
memex start includes the recording rule, Library location, pending trail
questions, and an inbox summary for the selected MEMEX_AGENT. Inside the
workspace it also includes selected MEMORY.md sections, owner preferences,
and Scribe receipts. This context has no strict total size cap.
Live recording is primary. Optional Scribe extraction reads Claude Code transcripts and appends qualifying missed records to existing Books. It cannot create Books, replace claims, write trails, or claim agreement. Its prompt asks for exact transcript evidence and preserved attribution. Tool results are not part of the spoken-text input, so a reported test result must remain a report.
The runner limits conversation chunks, output, daily calls, and retry frequency.
It records line and character cursors, caches completed extractions before
writing, and retries delivery without repeating a model call. Failures do not
advance the cursor. Unknown legacy coverage requires explicit recovery.
--dry makes no model call and writes no receipt. See Scribe.
Dispatch is local agent correspondence. Immutable JSON events
hold messages, acknowledgments, and file references outside brain/. Reading
does not acknowledge; only the recipient may acknowledge a message through the
CLI. Neither a handoff nor an acknowledgment proves a Library claim.
The summary index is rebuildable and checked against event contents. Inbox validation still reads the event collection; direct lookup reads one event. Sender names are declarations, not authenticated identities. Dispatch grants no permission and executes nothing from a message.
memex doctor reports parsing, schema, broken links, stale or removed index
entries, missing embeddings, legacy Observed sections, prompt/operation mismatch,
and duplicate trail steps. It exits nonzero on failed checks. It does not verify
truth, approval, retrieval quality, or backup restorability.
bin/memex-snapshot DESTINATION creates DESTINATION/memex/ outside the live
workspace while holding the runtime locks. It preserves Books, candidate state,
Scribe receipts, extraction caches, call budgets, queued jobs, and Dispatch.
The recall index is excluded because it can be rebuilt. Existing targets are
refused. The snapshot is plaintext: it is a preparation step, not an encrypted
backup or upload service.
For a personal Library, keep local version history and configure encrypted backups separately. A useful deployment pattern encrypts to a public key whose private half is held off-machine, uploads to independently controlled stores, and retains a readable restore runbook containing no secrets. Verify the recipient, upload results, and an actual restoration into a scratch workspace. This repository provisions none of that infrastructure or scheduling.
bin/library renders the Library; --serve runs an optional viewer bound to
127.0.0.1. Vault rendering uses passphrase-based encryption and requires the
optional cryptography package. This viewer is separate from CLI recall. There
is no MCP server or general remote Library API. Do not expose the viewer as one.
The core runtime uses Python, PyYAML, and POSIX filesystem primitives. Linux is the documented environment. Native Windows support, fixed-size session context, multimodal ingestion, and external connectors are not implemented.