Skip to content

MCP list and search tools return priority and metadata with different JSON types than kb_get #1503

Open

Description

@markramm

What happens. MCP read tools return the same field with different JSON types depending on the tool.

task_create {"kb_name":"emp","title":"Register domain","priority":7}
  -> {"created":true,"entry_id":"register-domain",...,"priority":7,...}
kb_find_by_status {"kb_name":"emp","status":"open"}
  -> {"entries":[{"id":"register-domain",...,"priority":"7",...}]}

Reproduced on current dev (0.26.3) over pyrite mcp --tier write. The hallway report also saw metadata returned as a JSON string by kb_find_by_status and kb_search, but as an object by kb_get, and kb_list_entries returning priority as a string.

What should happen. A field has one JSON type in every MCP result (and the CLI's --format json): priority an integer for tasks, metadata an object. An agent should not need per-tool parsing.

Likely files. The row-to-dict conversion for index rows in list/search tools (pyrite/server/mcp_server.py handlers for kb_find_by_status, kb_list_entries, kb_search, and the storage backend row mappers in pyrite/storage/backends/sqlite_backend.py / postgres_backend.py), compared with kb_get, which goes through the entry model.

Acceptance (properties).

  • For one entry, kb_get, kb_find_by_status, kb_list_entries and kb_search return the same JSON type for priority, importance and metadata.
  • A test creates a task with a priority and compares the types across those tools.

Source: feedback/2026-11-16-hallway-mcp-stdio-memory.md Friction 7 and 8 (PR #1497).

🤖 Filed by the maintainer's conductor agent from the 2026-11-16 hallway test (Claude Code)

Activity

  1. added this to the 0.26.4 milestone on Nov 16, 2026
  2. added
    bugSomething isn't working
    help wantedExtra attention is needed
    mcpMCP server and tools
    on Nov 16, 2026
  3. markramm commented on Nov 16, 2026

    @markramm
    CollaboratorAuthor

    Groom 2026-11-16

    Automated groom (architect agent). Checked against dev at 9d2342c (v0.26.3), MCP tools called in-process (_dispatch_tool, admin tier) in a scratch HOME. No open PR.

    Principle 8, and the read half of principle 3. An index row has no single projection. Raw-SQL readers hand back the JSON text column, the ORM readers decode it, and four consumers each carry their own _parse_metadata.

    The framing needs one correction: kb_get is not the integer reference. For a task file holding priority: 7:

    tool priority metadata
    task_create, task_list 7 (int) n/a
    kb_get "7" object
    kb_list_entries, kb_recent "7" object
    kb_find_by_status, kb_find_by_assignee "7" JSON string "{}"
    kb_search (keyword) absent JSON string

    kb_get mirrors the index's TEXT column on purpose (IndexManager.read_fields → _as_column, pyrite/storage/index.py about l.262, pinned by tests/test_get_reads_the_file.py::test_on_a_fresh_index_the_file_read_equals_the_row). The integer comes from the task service's coerce_task_priority. And this changes files: sending kb_get's priority back through kb_update rewrote priority: 7 to priority: "7" (reproduced). That is the open half of #730 P5 ("the fields of a read are the file's parsed values"), which kb/design.md, "Where the code does not match yet", already lists.

    So this is two problems with different weights:

    • A. metadata (and aliases) are JSON text on raw readers. A plain bug with one fix point. Groomed below for 0.26.4.
    • B. priority is text on every index read. priority is one TEXT column for a polymorphic field (a task's int 1-10, a backlog item's word high). Returning the file's type from list rows needs either a storage change or a cast by declared type. It belongs with B6 write path: phases, acceptance and footprint (spike result) #730 P5. Decision, not dispatchable as filed.

    The user's model. "A field has one JSON type whichever tool I read it with, and what I read I can write back unchanged."
    The implementation model. Every index read returns rows through one shaping function (decoded metadata object, aliases list, declared fields lifted), and a single-entry read returns the file's own values (P5).

    1. Contracts that apply

    • docs/json-contracts.md says nothing about the shape of an entry row in a list or search result. That contract is written nowhere: a finding, and a docs task (add a "row shape" section when A lands).
    • B6 write path: phases, acceptance and footprint (spike result) #730 P5: a read's fields are the file's parsed values; normalised values only under the derived key. It governs B.
    • kb/maps/surfaces-and-output-contracts.md and kb/maps/index-database-and-derived-state.md: the index is a cache, and its column types are not the file's.

    2. What the code does today

    • ORM readers go through pyrite/storage/backends/base_backend.py _entry_to_dict (about l.425), which decodes metadata (_parse_metadata, about l.411), parses aliases and lifts declared fields (lift_declared_fields). These serve kb_list_entries, kb_recent and kb_get's mirror.
    • Raw readers return dict(zip(cols, row)) with metadata undecoded:
      • pyrite/storage/queries.py: find_by_assignee (about l.383), find_overdue (l.428), find_by_status (l.466), find_by_location (l.499).
      • pyrite/storage/backends/sqlite_backend.py: search (FTS select, about l.265), the selects at about l.346 and l.368.
      • postgres_backend.py: about l.367, 459, 479, 553.
      • base_backend.py about l.1322.
      • Reproduced: kb_find_by_assignee returns metadata as a string and aliases as None.
    • Four copies of a metadata decoder exist, one per consumer: base_backend._parse_metadata, server/endpoints/collections.py _parse_metadata (l.34), services/task_service.py _parse_metadata (l.996), services/rubric_checkers.py _parse_metadata (l.74). That is the principle-8 signature.

    3. Invariant and surfaces

    Every entry row any index read returns has metadata as an object and aliases as a list, shaped by one function. (B, after the decision: a field's JSON type is the file's, on every read.)

    Enumeration source: the MCP tool table (pyrite/server/tool_schemas.py, 106 tools at admin tier). Classify every tool whose result carries entry rows (entries, results, events, tasks):

    • ORM, safe: kb_get, kb_list_entries, kb_recent, kb_batch_read (via get_entry; check).
    • Raw, guarded by A: kb_find_by_status, kb_find_by_assignee, kb_find_overdue, kb_find_by_location, kb_search (keyword; check the semantic and hybrid legs).
    • Own dicts, classify: task_*, sw_*, wiki_*, zettel_*, kb_backlinks.
    • Out of scope: investigation_* (experimental until 0.28).
    • CLI --format json for the same readers (pyrite search, the task and find commands); REST /api/search drops metadata in its response model (SearchResult), so check /api/entries lists.
    • Internal consumers of raw rows (rubric checkers, collections, task service) must accept a dict. Their parsers probably already do; verify each one.

    Shapes (each a test row): metadata as {}, as a nested object, as NULL; aliases as NULL, [] and a list; a declared field held in the bag (a backlog item's rank), which must be lifted the same way on every reader; SQLite and Postgres (Postgres in the pg marker job only).

    4. Options

    • A1 (recommended). One shape_entry_row(row) -> dict in pyrite/storage/ (beside _entry_to_dict, which then calls it too). Every raw reader returns through it, and the four consumer copies of _parse_metadata become tolerant no-ops or are deleted. Add a grep-style guard test: no dict(zip(cols, row in a reader that returns entry rows without the shaper.
    • A2. Shape at the surfaces (MCP or the service). Rejected: the CLI, REST and internal consumers keep the raw shape.
    • B options, for the maintainer: (i) cast on read by the declared field type in the same shaper (task priority → int when the text is an integer; derived, no storage change, but a quoted "7" in a file reads as 7); (ii) also store each column field's JSON value in the bag and lift it (exact, but needs an index rebuild); (iii) leave lists as text, fix only the single read (P5), and document that list rows carry the index's text. I recommend (i), with P5 doing the single read in the same release, so kb_get and lists move together.

    5. Open questions

    • Does any consumer depend on metadata being a string (string matching on the JSON)? Grep "metadata" in pyrite/services and the extensions before changing the storage layer.
    • Should the shaper also project away file_path and content_hash on list rows? (The hallway's "25 keys" complaint.) No: that is a separate projection question.

    6. Pointers

    base_backend._entry_to_dict; pyrite/schema/field_aliases.py lift_declared_fields; tests/test_get_reads_the_file.py (the read = row pin); tests/test_mcp_tool_registry_is_scoped.py (iterating the tool table).

    7. Checked versus assumed

    Ran: the table above (task_create, kb_get, kb_list_entries, kb_recent, kb_find_by_status, kb_find_by_assignee, kb_search, task_list), and the kb_get → kb_update round trip. Not probed: Postgres, semantic search, sw_*. This groom is wrong if a consumer relies on the string form. Then A needs a compatibility pass across those consumers first.

    Acceptance:   (A) for one task and one note with metadata, every tool in the "raw" and "ORM" classes returns metadata as an object and aliases as a list;
                  a test parametrised over those tools fails today on kb_find_by_status and kb_search.
    Regimes:      NULL metadata; an empty KB; Postgres (pg job).
    Shapes:       the metadata/aliases/bag-field shapes above × each reader.
    Touches:      existing: pyrite/storage/queries.py, pyrite/storage/backends/sqlite_backend.py, pyrite/storage/backends/postgres_backend.py,
                  pyrite/storage/backends/base_backend.py, pyrite/services/task_service.py, pyrite/services/rubric_checkers.py,
                  pyrite/server/endpoints/collections.py, docs/json-contracts.md (row shape)
                  new: tests/test_index_rows_have_one_shape.py, changelog.d fragment. About 10 files, +150/-60.
    Sequence:     independent of #1502 and #1505 (get_timeline selects no metadata). #1483 touches pyrite/storage/index.py and repository.py, not these readers.
    Model:        sonnet for A, from this groom; opus if B (i) is decided into it
    heavy:        no (Postgres runs in its own CI job)
    Cold read:    yes (pyrite/storage/ readers, and a public field type on MCP, CLI JSON and REST)
    Out of scope: B until decided; the investigation_* tools (experimental); list-row projection size.
    Decision:     B: how priority (and any TEXT column holding a typed file value) reads. Recommendation (i), shipped together with #730 P5's single read.
                  The counter-case: (ii) is exact and is the only option that reads a quoted "7" as a string.
    

    Hand-maintained files: A writes nothing. B fixes an existing hazard (a read sent back rewrites 7 as "7") and does not create one.

  4. markramm commented on Nov 16, 2026

    @markramm
    CollaboratorAuthor

    Decision (maintainer, 2026-11-16): B option (i): list and search rows cast a TEXT column value by the field's declared type in the one row shaper (A1), shipped with #730 P5's single read so a read written back never changes a file's type. (ii) stays available if a quoted value that must stay a string turns up.

    🤖 Recorded by the maintainer's conductor agent (Claude Code)

  5. markramm commented on Nov 17, 2026

    @markramm
    CollaboratorAuthor

    Claimed by the maintainer's loop, 2026-11-17: a cloud worker is building this from the groom above with the maintainer's decision (option A1, one row-shaping function every reader of entry rows returns through; option B (i), list and search rows cast a TEXT value by the field's declared type, and the single read moves with them). It posts its plan on a draft pull request before building; the draft will link here. Milestone 0.26.4. If you were about to start on it, say so here and the work can be shared or handed over.

    🤖 Written by the maintainer's conductor agent (Claude Code)

  6. markramm commented on Nov 17, 2026

    @markramm
    CollaboratorAuthor

    Claimed: draft pull request #1628.


    Generated by Claude Code

  7. markramm commented on Nov 17, 2026

    @markramm
    CollaboratorAuthor

    Decision (maintainer, 2026-11-17): (a). The task type's fields are declared in the schema in PR #1628, so the declared-type cast covers task priority on every reader.

    🤖 Recorded by the maintainer's conductor agent (Claude Code)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinghelp wantedExtra attention is neededmcpMCP server and tools

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions