Description
mempalace status reports the wrong drawer count on palaces with >10,000 drawers. No error, no warning — it silently displays 10000 drawers regardless of the actual count.
Root cause
miner.py:status() uses col.get(limit=10000, include=["metadatas"]) to fetch all drawer metadata, then displays len(metas) as the count. ChromaDB caps the result at 10,000 rows.
# miner.py line ~408
r = col.get(limit=10000, include=["metadatas"])
metas = r["metadatas"]
# ...
print(f" MemPalace Status — {len(metas)} drawers") # always ≤ 10000
Impact
- Displayed count is wrong. My palace has 122,686 drawers; status showed 10,000.
- Per-wing/room breakdown is incomplete. Only the first 10k drawers (by ChromaDB's internal ordering) are counted — some wings are entirely missing from the output.
- No warning that results are truncated. Users assume the count is accurate.
Fix
Paginate with offset and use col.count() for the total:
total = col.count()
metas = []
batch_size = 10000
offset = 0
while offset < total:
r = col.get(limit=batch_size, offset=offset, include=["metadatas"])
batch = r["metadatas"]
if not batch:
break
metas.extend(batch)
offset += len(batch)
# ...
print(f" MemPalace Status — {total} drawers")
Tested on a 122,686-drawer palace — correctly displays the full count and complete wing/room breakdown.
Related
Environment
- mempalace 3.0.14
- chromadb 0.6.3
- Python 3.13
- macOS
Description
mempalace statusreports the wrong drawer count on palaces with >10,000 drawers. No error, no warning — it silently displays10000 drawersregardless of the actual count.Root cause
miner.py:status()usescol.get(limit=10000, include=["metadatas"])to fetch all drawer metadata, then displayslen(metas)as the count. ChromaDB caps the result at 10,000 rows.Impact
Fix
Paginate with
offsetand usecol.count()for the total:Tested on a 122,686-drawer palace — correctly displays the full count and complete wing/room breakdown.
Related
mempalace statuscrashes with "too many SQL variables" on large palaces #802 —statuscrashes with "too many SQL variables" on large palaces (different symptom, same area)list_wings/get_taxonomytruncated at 10k in MCP tools (same root cause, different component)Environment