The daemon's memory keeps growing past 1 GB because glibc holds on to memory that SQLite has already freed

[dqtz5vpvj9-create]

agentsview version

main at d0395b5 (also v0.41.1 and v0.42.0)

Install method

Built from source (Linux, glibc malloc, mattn/go-sqlite3)

OS / platform

Linux x86_64 (Debian-based), 31 GB RAM, glibc 2.x

Which agent and version

Not agent-specific

Which model(s)

Not applicable

What happened, and what did you expect

Scope

One of three independent causes of a daemon footprint that tracks archive size instead of change rate; see #1584 (after-sync embedding refresh reads the whole corpus) and #1586 (tool-result text stored twice).

What happened

The serve daemon's RSS ratchets upward and never comes back down: 233 MB right after startup, 448 MB after 40 minutes, 1.18 GB after 22 hours on the previous build, with the same archive and no growth in the data it serves.

Splitting RSS by mapping (/proc/PID/smaps) shows the Go side is not the problem. runServe already installs debug.SetMemoryLimit(256 MiB) and the [anon: Go: heap] mappings hold about 33 MB. The growth is entirely in unnamed anonymous regions at 0x7f.., i.e. glibc malloc arenas used by the cgo SQLite driver: 331 MB at the 448 MB sample, against a theoretical SQLite page-cache ceiling of about 110 MB (4 archive readers, the writers, 4 usage-cache connections and 2 vector connections at _cache_size = -8192).

The mechanism is glibc's, not SQLite's: with about 20 threads each cgo call lands in a per-thread arena, freed chunks stay in the arena, and fragmentation across arenas means the high-water mark of every transient (usage rollups with temp_store=MEMORY, large queries, full parses) is retained forever. A reference-bit measurement (clear_refs + Referenced) over a two-minute idle window confirmed that after the fix below the remaining anonymous memory is 85 to 90% hot, so the retained portion was purely cold.

A/B measurement

Same binary, same archive, same load (five rounds of the usage summary, top sessions, comparison, session list and usage endpoints), only the allocator policy changed through the systemd unit:

	default glibc	MALLOC_ARENA_MAX=2 + MALLOC_TRIM_THRESHOLD_=1048576
RSS after startup (anon)	293 MB (237)	166 MB (111)
RSS after 3 load rounds (anon)	356 MB (300), stays	157 MB (101), flat
usage/summary latency	0.5 to 0.8 s	0.47 to 0.77 s

Expected behavior

The daemon's resident memory should follow its working set, not the high-water mark of past transients. The two-line allocator policy above achieves that with no measurable latency cost; the equivalent in code is mallopt(M_ARENA_MAX, 2) and mallopt(M_TRIM_THRESHOLD, 1<<20) at serve startup (Linux/glibc only), or a malloc_trim(0) next to the existing debug.FreeOSMemory() in the end-of-pass scavenge. Documenting the environment variables would already help operators on small machines.

Sample session file or snippet

RSS by mapping (448 MB sample):
  [unnamed anon 0x7b..]        331 MB   <- glibc arenas (SQLite via cgo)
  agentsview binary text        36 MB
  [anon: Go: heap]              33 MB
  [heap]                        12 MB
  [anon: Go: immortal metadata]  9 MB

Steps to reproduce

1. Run agentsview serve on Linux against a multi-GB archive and note RSS after startup.
2. Exercise the usage endpoints or let a few syncs happen; watch RSS climb and never return.
3. Restart with MALLOC_ARENA_MAX=2 MALLOC_TRIM_THRESHOLD_=1048576 in the environment; RSS stays flat under the same load.

Checklist

• I searched existing issues
• I removed secrets and private data from any attached session files