You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
MCP list and search tools return priority and metadata with different JSON types than kb_get #1503
Reproduced on current dev (0.26.3) over pyrite mcp --tier write. The hallway report also saw metadata returned as a JSON string by kb_find_by_status and kb_search, but as an object by kb_get, and kb_list_entries returning priority as a string.
What should happen. A field has one JSON type in every MCP result (and the CLI's --format json): priority an integer for tasks, metadata an object. An agent should not need per-tool parsing.
Likely files. The row-to-dict conversion for index rows in list/search tools (pyrite/server/mcp_server.py handlers for kb_find_by_status, kb_list_entries, kb_search, and the storage backend row mappers in pyrite/storage/backends/sqlite_backend.py / postgres_backend.py), compared with kb_get, which goes through the entry model.
Acceptance (properties).
For one entry, kb_get, kb_find_by_status, kb_list_entries and kb_search return the same JSON type for priority, importance and metadata.
A test creates a task with a priority and compares the types across those tools.
Automated groom (architect agent). Checked against dev at 9d2342c (v0.26.3), MCP tools called in-process (_dispatch_tool, admin tier) in a scratch HOME. No open PR.
Principle 8, and the read half of principle 3. An index row has no single projection. Raw-SQL readers hand back the JSON text column, the ORM readers decode it, and four consumers each carry their own _parse_metadata.
The framing needs one correction: kb_get is not the integer reference. For a task file holding priority: 7:
tool
priority
metadata
task_create, task_list
7 (int)
n/a
kb_get
"7"
object
kb_list_entries, kb_recent
"7"
object
kb_find_by_status, kb_find_by_assignee
"7"
JSON string"{}"
kb_search (keyword)
absent
JSON string
kb_get mirrors the index's TEXT column on purpose (IndexManager.read_fields → _as_column, pyrite/storage/index.py about l.262, pinned by tests/test_get_reads_the_file.py::test_on_a_fresh_index_the_file_read_equals_the_row). The integer comes from the task service's coerce_task_priority. And this changes files: sending kb_get's priority back through kb_update rewrote priority: 7 to priority: "7" (reproduced). That is the open half of #730 P5 ("the fields of a read are the file's parsed values"), which kb/design.md, "Where the code does not match yet", already lists.
So this is two problems with different weights:
A. metadata (and aliases) are JSON text on raw readers. A plain bug with one fix point. Groomed below for 0.26.4.
B. priority is text on every index read.priority is one TEXT column for a polymorphic field (a task's int 1-10, a backlog item's word high). Returning the file's type from list rows needs either a storage change or a cast by declared type. It belongs with B6 write path: phases, acceptance and footprint (spike result) #730 P5. Decision, not dispatchable as filed.
The user's model. "A field has one JSON type whichever tool I read it with, and what I read I can write back unchanged." The implementation model. Every index read returns rows through one shaping function (decoded metadata object, aliases list, declared fields lifted), and a single-entry read returns the file's own values (P5).
1. Contracts that apply
docs/json-contracts.md says nothing about the shape of an entry row in a list or search result. That contract is written nowhere: a finding, and a docs task (add a "row shape" section when A lands).
kb/maps/surfaces-and-output-contracts.md and kb/maps/index-database-and-derived-state.md: the index is a cache, and its column types are not the file's.
2. What the code does today
ORM readers go through pyrite/storage/backends/base_backend.py_entry_to_dict (about l.425), which decodes metadata (_parse_metadata, about l.411), parses aliases and lifts declared fields (lift_declared_fields). These serve kb_list_entries, kb_recent and kb_get's mirror.
Raw readers return dict(zip(cols, row)) with metadata undecoded:
pyrite/storage/backends/sqlite_backend.py: search (FTS select, about l.265), the selects at about l.346 and l.368.
postgres_backend.py: about l.367, 459, 479, 553.
base_backend.py about l.1322.
Reproduced: kb_find_by_assignee returns metadata as a string and aliases as None.
Four copies of a metadata decoder exist, one per consumer: base_backend._parse_metadata, server/endpoints/collections.py_parse_metadata (l.34), services/task_service.py_parse_metadata (l.996), services/rubric_checkers.py_parse_metadata (l.74). That is the principle-8 signature.
3. Invariant and surfaces
Every entry row any index read returns has metadata as an object and aliases as a list, shaped by one function. (B, after the decision: a field's JSON type is the file's, on every read.)
Enumeration source: the MCP tool table (pyrite/server/tool_schemas.py, 106 tools at admin tier). Classify every tool whose result carries entry rows (entries, results, events, tasks):
Raw, guarded by A:kb_find_by_status, kb_find_by_assignee, kb_find_overdue, kb_find_by_location, kb_search (keyword; check the semantic and hybrid legs).
Own dicts, classify:task_*, sw_*, wiki_*, zettel_*, kb_backlinks.
Out of scope:investigation_* (experimental until 0.28).
CLI --format json for the same readers (pyrite search, the task and find commands); REST /api/search drops metadata in its response model (SearchResult), so check /api/entries lists.
Internal consumers of raw rows (rubric checkers, collections, task service) must accept a dict. Their parsers probably already do; verify each one.
Shapes (each a test row): metadata as {}, as a nested object, as NULL; aliases as NULL, [] and a list; a declared field held in the bag (a backlog item's rank), which must be lifted the same way on every reader; SQLite and Postgres (Postgres in the pg marker job only).
4. Options
A1 (recommended). One shape_entry_row(row) -> dict in pyrite/storage/ (beside _entry_to_dict, which then calls it too). Every raw reader returns through it, and the four consumer copies of _parse_metadata become tolerant no-ops or are deleted. Add a grep-style guard test: no dict(zip(cols, row in a reader that returns entry rows without the shaper.
A2. Shape at the surfaces (MCP or the service). Rejected: the CLI, REST and internal consumers keep the raw shape.
B options, for the maintainer: (i) cast on read by the declared field type in the same shaper (task priority → int when the text is an integer; derived, no storage change, but a quoted "7" in a file reads as 7); (ii) also store each column field's JSON value in the bag and lift it (exact, but needs an index rebuild); (iii) leave lists as text, fix only the single read (P5), and document that list rows carry the index's text. I recommend (i), with P5 doing the single read in the same release, so kb_get and lists move together.
5. Open questions
Does any consumer depend on metadata being a string (string matching on the JSON)? Grep "metadata" in pyrite/services and the extensions before changing the storage layer.
Should the shaper also project away file_path and content_hash on list rows? (The hallway's "25 keys" complaint.) No: that is a separate projection question.
6. Pointers
base_backend._entry_to_dict; pyrite/schema/field_aliases.pylift_declared_fields; tests/test_get_reads_the_file.py (the read = row pin); tests/test_mcp_tool_registry_is_scoped.py (iterating the tool table).
7. Checked versus assumed
Ran: the table above (task_create, kb_get, kb_list_entries, kb_recent, kb_find_by_status, kb_find_by_assignee, kb_search, task_list), and the kb_get → kb_update round trip. Not probed: Postgres, semantic search, sw_*. This groom is wrong if a consumer relies on the string form. Then A needs a compatibility pass across those consumers first.
Acceptance: (A) for one task and one note with metadata, every tool in the "raw" and "ORM" classes returns metadata as an object and aliases as a list;
a test parametrised over those tools fails today on kb_find_by_status and kb_search.
Regimes: NULL metadata; an empty KB; Postgres (pg job).
Shapes: the metadata/aliases/bag-field shapes above × each reader.
Touches: existing: pyrite/storage/queries.py, pyrite/storage/backends/sqlite_backend.py, pyrite/storage/backends/postgres_backend.py,
pyrite/storage/backends/base_backend.py, pyrite/services/task_service.py, pyrite/services/rubric_checkers.py,
pyrite/server/endpoints/collections.py, docs/json-contracts.md (row shape)
new: tests/test_index_rows_have_one_shape.py, changelog.d fragment. About 10 files, +150/-60.
Sequence: independent of #1502 and #1505 (get_timeline selects no metadata). #1483 touches pyrite/storage/index.py and repository.py, not these readers.
Model: sonnet for A, from this groom; opus if B (i) is decided into it
heavy: no (Postgres runs in its own CI job)
Cold read: yes (pyrite/storage/ readers, and a public field type on MCP, CLI JSON and REST)
Out of scope: B until decided; the investigation_* tools (experimental); list-row projection size.
Decision: B: how priority (and any TEXT column holding a typed file value) reads. Recommendation (i), shipped together with #730 P5's single read.
The counter-case: (ii) is exact and is the only option that reads a quoted "7" as a string.
Hand-maintained files: A writes nothing. B fixes an existing hazard (a read sent back rewrites 7 as "7") and does not create one.
Decision (maintainer, 2026-11-16): B option (i): list and search rows cast a TEXT column value by the field's declared type in the one row shaper (A1), shipped with #730 P5's single read so a read written back never changes a file's type. (ii) stays available if a quoted value that must stay a string turns up.
🤖 Recorded by the maintainer's conductor agent (Claude Code)
Claimed by the maintainer's loop, 2026-11-17: a cloud worker is building this from the groom above with the maintainer's decision (option A1, one row-shaping function every reader of entry rows returns through; option B (i), list and search rows cast a TEXT value by the field's declared type, and the single read moves with them). It posts its plan on a draft pull request before building; the draft will link here. Milestone 0.26.4. If you were about to start on it, say so here and the work can be shared or handed over.
🤖 Written by the maintainer's conductor agent (Claude Code)
Decision (maintainer, 2026-11-17): (a). The task type's fields are declared in the schema in PR #1628, so the declared-type cast covers task priority on every reader.
🤖 Recorded by the maintainer's conductor agent (Claude Code)
Activity
markramm commentedon Nov 16, 2026
Groom 2026-11-16
Automated groom (architect agent). Checked against
devat 9d2342c (v0.26.3), MCP tools called in-process (_dispatch_tool, admin tier) in a scratch HOME. No open PR.Principle 8, and the read half of principle 3. An index row has no single projection. Raw-SQL readers hand back the JSON text column, the ORM readers decode it, and four consumers each carry their own
_parse_metadata.The framing needs one correction:
kb_getis not the integer reference. For a task file holdingpriority: 7:prioritymetadatatask_create,task_list7(int)kb_get"7"kb_list_entries,kb_recent"7"kb_find_by_status,kb_find_by_assignee"7""{}"kb_search(keyword)kb_getmirrors the index's TEXT column on purpose (IndexManager.read_fields→_as_column,pyrite/storage/index.pyabout l.262, pinned bytests/test_get_reads_the_file.py::test_on_a_fresh_index_the_file_read_equals_the_row). The integer comes from the task service'scoerce_task_priority. And this changes files: sendingkb_get'spriorityback throughkb_updaterewrotepriority: 7topriority: "7"(reproduced). That is the open half of #730 P5 ("the fields of a read are the file's parsed values"), whichkb/design.md, "Where the code does not match yet", already lists.So this is two problems with different weights:
metadata(andaliases) are JSON text on raw readers. A plain bug with one fix point. Groomed below for 0.26.4.priorityis text on every index read.priorityis one TEXT column for a polymorphic field (a task's int 1-10, a backlog item's wordhigh). Returning the file's type from list rows needs either a storage change or a cast by declared type. It belongs with B6 write path: phases, acceptance and footprint (spike result) #730 P5. Decision, not dispatchable as filed.The user's model. "A field has one JSON type whichever tool I read it with, and what I read I can write back unchanged."
The implementation model. Every index read returns rows through one shaping function (decoded
metadataobject,aliaseslist, declared fields lifted), and a single-entry read returns the file's own values (P5).1. Contracts that apply
docs/json-contracts.mdsays nothing about the shape of an entry row in a list or search result. That contract is written nowhere: a finding, and a docs task (add a "row shape" section when A lands).kb/maps/surfaces-and-output-contracts.mdandkb/maps/index-database-and-derived-state.md: the index is a cache, and its column types are not the file's.2. What the code does today
pyrite/storage/backends/base_backend.py_entry_to_dict(about l.425), which decodesmetadata(_parse_metadata, about l.411), parsesaliasesand lifts declared fields (lift_declared_fields). These servekb_list_entries,kb_recentandkb_get's mirror.dict(zip(cols, row))withmetadataundecoded:pyrite/storage/queries.py:find_by_assignee(about l.383),find_overdue(l.428),find_by_status(l.466),find_by_location(l.499).pyrite/storage/backends/sqlite_backend.py:search(FTS select, about l.265), the selects at about l.346 and l.368.postgres_backend.py: about l.367, 459, 479, 553.base_backend.pyabout l.1322.kb_find_by_assigneereturnsmetadataas a string andaliasesasNone.base_backend._parse_metadata,server/endpoints/collections.py_parse_metadata(l.34),services/task_service.py_parse_metadata(l.996),services/rubric_checkers.py_parse_metadata(l.74). That is the principle-8 signature.3. Invariant and surfaces
Every entry row any index read returns has
metadataas an object andaliasesas a list, shaped by one function. (B, after the decision: a field's JSON type is the file's, on every read.)Enumeration source: the MCP tool table (
pyrite/server/tool_schemas.py, 106 tools at admin tier). Classify every tool whose result carries entry rows (entries,results,events,tasks):kb_get,kb_list_entries,kb_recent,kb_batch_read(viaget_entry; check).kb_find_by_status,kb_find_by_assignee,kb_find_overdue,kb_find_by_location,kb_search(keyword; check the semantic and hybrid legs).task_*,sw_*,wiki_*,zettel_*,kb_backlinks.investigation_*(experimental until 0.28).--format jsonfor the same readers (pyrite search, the task and find commands); REST/api/searchdropsmetadatain its response model (SearchResult), so check/api/entrieslists.Shapes (each a test row):
metadataas{}, as a nested object, as NULL;aliasesas NULL,[]and a list; a declared field held in the bag (a backlog item'srank), which must be lifted the same way on every reader; SQLite and Postgres (Postgres in thepgmarker job only).4. Options
shape_entry_row(row) -> dictinpyrite/storage/(beside_entry_to_dict, which then calls it too). Every raw reader returns through it, and the four consumer copies of_parse_metadatabecome tolerant no-ops or are deleted. Add a grep-style guard test: nodict(zip(cols, rowin a reader that returnsentryrows without the shaper.priority→ int when the text is an integer; derived, no storage change, but a quoted"7"in a file reads as 7); (ii) also store each column field's JSON value in the bag and lift it (exact, but needs an index rebuild); (iii) leave lists as text, fix only the single read (P5), and document that list rows carry the index's text. I recommend (i), with P5 doing the single read in the same release, sokb_getand lists move together.5. Open questions
metadatabeing a string (string matching on the JSON)? Grep"metadata"inpyrite/servicesand the extensions before changing the storage layer.file_pathandcontent_hashon list rows? (The hallway's "25 keys" complaint.) No: that is a separate projection question.6. Pointers
base_backend._entry_to_dict;pyrite/schema/field_aliases.pylift_declared_fields;tests/test_get_reads_the_file.py(the read = row pin);tests/test_mcp_tool_registry_is_scoped.py(iterating the tool table).7. Checked versus assumed
Ran: the table above (
task_create,kb_get,kb_list_entries,kb_recent,kb_find_by_status,kb_find_by_assignee,kb_search,task_list), and thekb_get→kb_updateround trip. Not probed: Postgres, semantic search,sw_*. This groom is wrong if a consumer relies on the string form. Then A needs a compatibility pass across those consumers first.Hand-maintained files: A writes nothing. B fixes an existing hazard (a read sent back rewrites
7as"7") and does not create one.markramm commentedon Nov 16, 2026
Decision (maintainer, 2026-11-16): B option (i): list and search rows cast a TEXT column value by the field's declared type in the one row shaper (A1), shipped with #730 P5's single read so a read written back never changes a file's type. (ii) stays available if a quoted value that must stay a string turns up.
🤖 Recorded by the maintainer's conductor agent (Claude Code)
markramm commentedon Nov 17, 2026
Claimed by the maintainer's loop, 2026-11-17: a cloud worker is building this from the groom above with the maintainer's decision (option A1, one row-shaping function every reader of entry rows returns through; option B (i), list and search rows cast a TEXT value by the field's declared type, and the single read moves with them). It posts its plan on a draft pull request before building; the draft will link here. Milestone 0.26.4. If you were about to start on it, say so here and the work can be shared or handed over.
🤖 Written by the maintainer's conductor agent (Claude Code)
markramm commentedon Nov 17, 2026
Claimed: draft pull request #1628.
Generated by Claude Code
test: a field has one JSON type on every reader of entry rows (#1503)
fix: every storage reader returns entry rows through one shaper (#1503)
docs: the entry row shape every reader returns, and the fragment (#1503)
fix: the row shaper adds no key; the lift stays with the ORM row (#1503)
refactor: the single read casts a column by its declared type (#1503)
test: mark the negative controls in the row-shape tests (#1503)
refactor: type the shaper's calls so mypy adds no error (#1503)
markramm commentedon Nov 17, 2026
Decision (maintainer, 2026-11-17): (a). The
tasktype's fields are declared in the schema in PR #1628, so the declared-type cast covers task priority on every reader.🤖 Recorded by the maintainer's conductor agent (Claude Code)
fix: cast only plain ASCII numbers; a task declares its priority a nu…
fix: qa validate names text in a field declared a number (#1503)
test: backend cells seeded as the indexer binds; the structural guard…
docs: the row shape says what the strict cast and a write-back do (#1503
test: a REST GET put back whole leaves the file byte-identical (#1503)
refactor: type TASK_TYPE_METADATA so mypy adds no error (#1503)
fix: qa validate's number_as_text says what the reader returns (#1503)
fix: REST's typed date and status fields carry a cast value (#1503)
docs: state every write-back exception the readers make, each pinned (#…