Describe the bug
Repeated explicit Memory appends have superlinear persistent-storage growth and steadily increasing write latency, even when each entry contains only a few bytes of text.
Every Memory Revision stores the complete current manifest. Appending one entry therefore writes manifests of size 1, 2, 3, ... N across the immutable revision history. Entry bodies are not duplicated, but the cumulative manifest directory approaches quadratic growth.
There is currently no maximum entry count, manifest-byte budget, automatic Memory split, or tombstone compaction boundary. The unpaginated list_memory_entries path also materializes the complete current manifest and all referenced entry versions.
The Memory RFC identifies split and compaction thresholds as future work, but the current service can grow past practical limits without an explicit error or degradation signal.
Steps to reproduce
At commit a6e0dfe:
- Create a fresh SQLite-backed Server with Dashboard, MCP, metrics, and access logging disabled.
- Repeatedly call POST /v1/memory/remember in one scope with one short unique fact per request.
- Close the Server.
- Run PRAGMA wal_checkpoint(TRUNCATE) and VACUUM.
- Record the database size and average duration of the final 20 writes.
- Repeat with 50, 100, and 200 entries.
Observed:
| Entries |
SQLite database bytes |
Average final 20 writes |
| 50 |
671,744 |
40.57 ms |
| 100 |
1,634,304 |
65.21 ms |
| 200 |
5,222,400 |
111.81 ms |
The test body was only bounded fact NNNN.
Relevant paths:
- src/powercontext/builtin/artifacts/memory/models.py:62-65
- src/powercontext/builtin/artifacts/memory/service.py:699-731
- src/powercontext/builtin/persistence/artifacts.py:243-262
- src/powercontext/builtin/persistence/memory.py:157-175
- docs/en/rfcs/0014_memory_layer_design.md, especially the deferred split/compaction thresholds
Expected behavior
A long-lived Memory service needs an explicit, observable capacity contract.
At minimum:
- Enforce a current-manifest entry or byte budget before unbounded growth.
- Return a stable error or route new entries to another Memory when the budget is reached.
- Paginate or otherwise bound list_memory_entries and complete-history operations.
- Define and test split and inactive-tombstone compaction behavior.
- Add a scale regression test that measures storage and write amplification at increasing entry counts.
The exact split/compaction design may require an RFC, but the current API should not accept growth indefinitely without a budget.
Actual behavior
Each append reconstructs and persists the complete manifest in a new immutable Artifact Revision. Storage and write work increase with the number of prior entries, with no configured ceiling or automatic split.
Environment
- PowerContext: 0.0.3.dev11+ga6e0dfe8b.d20260823
- Commit: a6e0dfe
- Python: 3.14.3
- Backend: SQLite with WAL, checkpointed and vacuumed before measurement
- OS: Windows 11
- External inference and embedding providers: disabled
Are you willing to submit a PR to fix this bug?
Describe the bug
Repeated explicit Memory appends have superlinear persistent-storage growth and steadily increasing write latency, even when each entry contains only a few bytes of text.
Every Memory Revision stores the complete current manifest. Appending one entry therefore writes manifests of size 1, 2, 3, ... N across the immutable revision history. Entry bodies are not duplicated, but the cumulative manifest directory approaches quadratic growth.
There is currently no maximum entry count, manifest-byte budget, automatic Memory split, or tombstone compaction boundary. The unpaginated list_memory_entries path also materializes the complete current manifest and all referenced entry versions.
The Memory RFC identifies split and compaction thresholds as future work, but the current service can grow past practical limits without an explicit error or degradation signal.
Steps to reproduce
At commit a6e0dfe:
Observed:
The test body was only bounded fact NNNN.
Relevant paths:
Expected behavior
A long-lived Memory service needs an explicit, observable capacity contract.
At minimum:
The exact split/compaction design may require an RFC, but the current API should not accept growth indefinitely without a budget.
Actual behavior
Each append reconstructs and persists the complete manifest in a new immutable Artifact Revision. Storage and write work increase with the number of prior entries, with no configured ceiling or automatic split.
Environment
Are you willing to submit a PR to fix this bug?