Skip to content

Content addressed records#20

Merged
isTravis merged 13 commits into
mainfrom
tr/jsonl
Jun 9, 2026
Merged

Content addressed records#20
isTravis merged 13 commits into
mainfrom
tr/jsonl

Conversation

@isTravis

@isTravis isTravis commented Jun 9, 2026

Copy link
Copy Markdown
Member

Content-addressed records, a single push protocol, CLI foundation, and protocol documentation.

What changed

Content-addressed record storage — Records are now stored by their SHA-256 hash in a record_objects table, linked to versions via a version_records join table. Same record in 10 versions = stored once. This matches how files and schemas already worked. Removes the old records table entirely.

Single push protocol — Consolidated three push paths (simple push, chunked upload, negotiate push) into one git-like flow: negotiate → send records → commit. The client sends a manifest of record hashes, the server says what it needs, the client sends only those records in JSONL batches, then commits. 100,000 unchanged records + 5 new ones = only 5 transferred. Deletes uploads.ts (878 lines) and the push() function from versions.ts (~480 lines).

Record provenance — New /api/records/:hash/provenance endpoint traces a record back to every collection and version that includes it, with a first-seen date.

Batch record fetch — New POST /api/records/batch endpoint returns records by hash as a JSONL stream. Enables efficient pull sync.

CLI — Local underlay CLI with init, schema-set, add, status, commit, log, diff, remote, push, pull. Local versioning with the same content-addressed storage model. Push/pull use the negotiate protocol.

Protocol page (/protocol) — Full protocol spec: data model, hashing, push/pull, schemas, files, provenance, collaboration.

Updated docs — ai.txt, integration guide, API docs, and dev tool all updated for the single negotiate protocol.

@isTravis isTravis merged commit 2e6eaf9 into main Jun 9, 2026
1 check failed
@isTravis isTravis deleted the tr/jsonl branch June 9, 2026 01:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant