Let config.toml choose how much of each session the archive keeps - #1564
Let config.toml choose how much of each session the archive keeps#1564naveenspark wants to merge 1 commit into
Conversation
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
|
looking |
roborev: Combined Review (
|
b59259a to
eef1ebe
Compare
roborev: Combined Review (
|
eef1ebe to
19ff7f5
Compare
roborev: Combined Review (
|
19ff7f5 to
95f293b
Compare
roborev: Combined Review (
|
95f293b to
e1dab18
Compare
roborev: Combined Review (
|
e1dab18 to
5a7ac56
Compare
roborev: Combined Review (
|
5a7ac56 to
806900c
Compare
roborev: Combined Review (
|
806900c to
a823321
Compare
roborev: Combined Review (
|
a823321 to
5dd57ee
Compare
roborev: Combined Review (
|
5dd57ee to
1ee3d47
Compare
roborev: Combined Review (
|
1ee3d47 to
e9752c0
Compare
roborev: Combined Review (
|
e9752c0 to
97c7622
Compare
roborev: Combined Review (
|
97c7622 to
8f31fd7
Compare
roborev: Combined Review (
|
8f31fd7 to
e0c3854
Compare
roborev: Combined Review (
|
e0c3854 to
6a53660
Compare
roborev: Combined Review (
|
6a53660 to
47ece0d
Compare
roborev: Combined Review (
|
47ece0d to
61e9f52
Compare
roborev: Combined Review (
|
61e9f52 to
e4eb810
Compare
roborev: Combined Review (
|
Operators who only want token and cost reports, and operators whose archive is dominated by tool output, both need to store less than the full transcript. `archive_content` in config.toml selects one of three policies: "full" (default), "transcripts", or "usage". `AGENTSVIEW_ARCHIVE_CONTENT` sets the policy when the config file does not, for a dedicated reporting archive in its own data directory. The transcripts policy keeps message and thinking text, titles, and tool call metadata while dropping tool inputs and results, which on a large archive are most of the file. The usage policy keeps only the session and message rows that token and cost reports need, classifies automation while the source text is still available, guards partial OpenCode sources against erasing retained usage, and settles transcript-derived signal and secret state so startup backfills do not revisit rows. The database handle is the single authority for the policy. Open variants and the sync engine only tighten it. Every write path projects sessions and messages before rows are written, and resync copies archived and trashed sessions through the same projection in SQL because those copies bypass the write path. Signals and secret findings are computed from the projected messages so a recompute from stored rows reproduces them. A usage rebuild also leaves cached insights and recall entries behind, since both hold transcript-derived text. OpenCode messages now record their storage message ID as source_uuid so archive guards match rows by identity instead of ordinal. The data version moves to 98 so existing rows are re-parsed to carry it. An existing archive is not rewritten when the policy changes; a full resync applies it to every session, including ones whose source files are gone. Squashed commits: - Add usage-only archive storage - stabilize vector shutdown cancellation test - preserve usage-only mode in no-sync daemon - perf(storage): compact usage-only archives - fix(storage): preserve usage-only reporting semantics - fix(storage): preserve subagent usage links - fix(sync): preserve compact usage history - fix(storage): settle compact signal state - fix(storage): satisfy compact call lint - fix(storage): guard optional message sanitization - Merge remote-tracking branch 'upstream/main' into feature/usage-only-storage - feat(storage): choose archive content policy in config.toml - fix(storage): match OpenCode rows by message ID and skip derived text Co-authored-by: Wes McKinney <wesm@users.noreply.github.com> Generated with Claude Code Co-authored-by: Claude <noreply@anthropic.com>
e4eb810 to
4b525fe
Compare
roborev: Combined Review (
|
Adds
archive_contenttoconfig.tomlso an archive can keep less than everything.What you get
transcriptskeeps the conversation and drops tool inputs and outputs. Search, transcript viewing, tool analytics, and the session tree keep working. On a 43 GB archive, tool payloads were 33 GB.usagekeeps only what token and cost reports need. About 41k sessions fit in 2.5 GB in the contributor's canary.AGENTSVIEW_ARCHIVE_CONTENTcovers a reporting archive in its own data directory.Applying it to an existing archive
Set the key, restart the daemon, run
agentsview sync --full. Back up first: dropped content cannot be recovered without the source files.What
transcriptsis notA storage policy, not a redaction guarantee. It removes the tool payload tables and structured inputs, drops rows and summaries that parsers mark as tool output, and rewrites the tool summaries parsers inline into message text. Text a provider wrote into a message body in its own format stays as written.
usagestores no text at all.Costs
transcripts, checks that read tool payloads see nothing, and archived sessions from RooCode, Kilo Legacy, gptme, OpenHands, and Aider parsed by earlier releases lose some text on rebuild, since those parsers mixed tool output into ordinary rows.usage, insights, recall, eval trajectories, and pin notes are refused or dropped.