Skip to content

importance: a create coerces what an update refuses (10.7 stored as 10, true as 1), and seven smaller limits left by #1570 #1588

Open

Description

@markramm

Follow-up to #1570 (fix for #1537), which makes an update that sets importance, and pyrite add, refuse a value outside 1 to 10 on every entry type. These are the answers #1570 leaves as stated limits. None leaves a number outside 1 to 10 in a file. Each row ran on #1570's head d508760 and on dev 0d4bcd4 unless it says otherwise.

A create coerces what an update refuses

pyrite create ... -f importance=<value> stored the same value on update -f (after #1570)
10.7 importance: 10 refused, VALIDATION_FAILED
3.9 importance: 3 accepted, written as 3.9
true importance: 1 refused
high, abc, blank the default (5) refused
11.2 refused refused

MCP kb_create does the same. The create exits 0 and says nothing about the change. The cut happens in the model before the range check (pyrite/models/, where importance is read with int()), so the check sees a value already in range.

What it should do (to decide): (a) a create refuses what an update refuses, with the same errors row (recommended: one rule for one field on every write); or (b) a create keeps coercing and reports it as a warning in its answer. Either way true is not 1.

The rest, one line each

  1. update -f importance=5.5 writes 5.5 as given: a number inside 1 to 10 is accepted whether or not it is whole. Decide: whole numbers only, or say in docs/json-contracts.md that a decimal is allowed.
  2. Whole-number text is whatever Python's int() reads. Over MCP kb_update, "10" (fullwidth digits), "1_0", "+5" and " 5\n" are accepted and written to the file as strings; a YAML reader outside Pyrite then sees text, not a number.
  3. The CLI cannot clear importance: -f importance= and -f importance=null are text and are refused; only MCP kb_update with "importance": null clears it.
  4. A file whose importance is a !!binary value: an update of another field keeps it with no "kept" warning (pyrite add refuses it for another reason).
  5. REST answers a non-number importance with FastAPI's top-level 422, not the VALIDATION_FAILED body the CLI and MCP give (read, not run).
  6. The "violation already on disk; kept" warning row carries rule: model_validation, where the refusal of the same value carries rule: range.
  7. pyrite qa validate reads the index's coerced value, so it says nothing about a file value an update now warns about.
  8. importance: !!str 7 in a file gets a row although "7" is accepted on a write.

Acceptance

  • The value table in tests/test_importance_range_on_every_type.py (the KINDS rows) gains a create column whose verdicts are written by hand from the rule chosen above, and the rows for items 1 to 4 and 6 to 8 stop being "stated limit".
  • The changelog fragment names every answer that changes.

Related: #1584 (a 30-digit importance already in a file). After #1570 merges. The decisions (create: refuse or report; decimals; item 2) are the maintainer's; the rest is small.

Activity

  1. markramm commented on Nov 17, 2026

    @markramm
    CollaboratorAuthor

    Decisions (maintainer, 2026-11-17): (a) a create refuses what an update refuses, with the same errors row; true is not 1. Importance is a whole number: 5.5 is refused. Text that is not plain ASCII digits ("10", "1_0", "+5", padded text) is refused, never written to the file as a string. The remaining one-liners stay as listed for whoever takes it.

    🤖 Recorded by the maintainer's conductor agent (Claude Code)

  2. added this to the 0.26.4 milestone on Nov 17, 2026
  3. markramm commented on Nov 17, 2026

    @markramm
    CollaboratorAuthor

    Automated groom (an AI agent working for the maintainer). Reproduced on dev at f0b010a. Written so someone new to Pyrite can take it from this issue alone.

    Groom 2026-11-17

    Dispatchable. The maintainer's decisions (comment above) are built in: a create refuses what an update refuses, with the same errors row; true is not 1; importance is a whole number (5.5 refused); text that is not plain ASCII digits is refused. One small question is left open in section 5, with a recommendation the worker can follow.

    Principle at stake: 8 (each rule lives in one place). The user's model: "importance is a whole number from 1 to 10. Whatever I send, through any command, is stored as I sent it or refused with the same answer." The implementation model: one predicate, validate_importance, judges the value as given. The model must keep that value long enough for Entry.validation_errors to see it on every path: create, add, update, and either spelling in the file. Today the model turns the value into an int first (safe_int), and only an update of a top-level key still has the original to judge.

    1. Contracts that apply

    • kb/design.md principle 8: the rule sits on Entry.validation_errors (pyrite/models/base.py), which the one write gate KBService._validate_write (pyrite/services/kb_service.py) applies (fix: a write refuses importance outside 1-10 on every type (#1537) #1570). It stays there; no new call sites.
    • ADR-0042 decision 8: a value already on disk in a field the write did not touch is kept, with a warning. The "file" verdicts in the test table keep that meaning.
    • docs/json-contracts.md, error shape: a refusal carries errors rows {field, rule, message, expected}. The importance rule itself is written in no doc (grep docs/ for 1-10: no hit). Add one sentence to docs/json-contracts.md next to the validation rows.
    • tests/test_importance_range_on_every_type.py (fix: a write refuses importance outside 1-10 on every type (#1537) #1570): one row per kind of value, with verdicts written by hand. Extend it; do not start a new table.

    2. What the code does today

    Reproduced (checked, ran on dev, a throwaway KB): pyrite create -k m -t note --title X -b x -f importance=<v> --format json exits 0 for every one of 10.7, 3.9, true, high, blank, 5.5, +5, 1_0, 05. The files then hold 10, 3, 1, 5, 5, 5, 5, 10, 5. update -f importance=10.7 is refused (VALIDATION_FAILED, rule: range).

    Surprises first:

    • The rule is also bypassed on update when the file spells importance under metadata: (not in the issue; the reader: a managed list field spelled only under metadata: (links, sources) is not read as the field, contrary to ADR-0051 #1025 nested spelling). On a file with metadata: {importance: 5}, update -f importance=10.7 and then -f importance=5.5 are both accepted and written as metadata.importance: 10.7 / 5.5. A file holding metadata: {importance: 10.7} gets no "kept" warning on a body update (checked, ran). The cause: Entry.validation_errors reads self.unrepresented_frontmatter().get("importance", self.importance). unrepresented_frontmatter is keyed by the file's top-level keys, so a nested value reaches the check only as the model's coerced int. Repo data: 1 file in kb/ uses the nested spelling (kb/notes/pyrite-user-personas-five-surfaces-five-mental-models.md); 367 use the top level, all whole numbers 3 to 8 (grep).
    • Why a create coerces: every entry class reads base fields through Entry._base_kwargs (pyrite/models/base.py), which sets "importance": safe_int(meta.get("importance"), 5) (pyrite/utils/parse.py safe_int: int(value) or the default). A create builds through build_entry (pyrite/models/factory.py) → from_frontmatter → _base_kwargs and has no source frontmatter, so unrepresented_frontmatter() is empty and the check sees 10, 1, 5. All 40 from_frontmatter implementations, in core and the five in-tree extensions, call _base_kwargs or super() (grep count per file), so that one reader is the place to keep the value as given.
    • validate_importance (pyrite/schema/validators.py): bool → False; int/float compared as is (so 5.5 passes); str via int() (so "10", "1_0", "+5", " 5\n" pass); anything else via float(), so bytes pass when they spell a number. Checked (ran): importance: !!binary NQ== (b"5") gives no row on a body update, while !!binary aGVsbG8= gets the kept warning. !!str 7 loads as a ruamel TaggedScalar, float() refuses it, and it gets a row (checked, ran).
    • The "kept" warning row hard-codes "rule": "model_validation" in KBService._validate_write, while the refusal row has the model's own rule (range) (checked, ran: o99 and s7 above). model_validation is pinned in 3 test files (tests/test_update_validates_like_create.py, tests/test_update_refuses_only_what_it_causes.py, tests/test_lifecycle_validation.py) and in no doc.
    • REST: CreateEntryRequest.importance and UpdateEntryRequest.importance (pyrite/server/schemas.py) are int | None = Field(None, ge=1, le=10). Pydantic coerces before the service: True→1, "1_0"→10, "+5"→5, " 5 "→5, 10.0→10. 10.7 → FastAPI's top-level 422 (checked, ran, on an equivalent pydantic model in-process; the route itself not run). So REST stores true as 1 on create and update today, which is against the decision.
    • CLI -f text is parsed by _parse_field_value (pyrite/cli/entry_commands.py), which calls int(stripped) first. On the CLI +5, 1_0, 05, 5 therefore arrive as the int 5/10 before any rule sees them, for every field, not just importance. See "Out of scope".
    • MCP tool schemas declare importance as integer (pyrite/server/tool_schemas.py, kb_create/kb_update), but _dispatch_tool does not enforce them (the issue's MCP rows). Whether a client enforces them is up to the client.

    3. Invariant and surfaces

    Invariant: for any value, by any surface, a create, add, bulk create and update of importance either store a whole number 1 to 10 as sent or refuse with the same errors row; a value already in the file is judged by the same predicate and only reported.

    • CLI create -f, update -f, add: coerces (ran) / judges / judges.
    • MCP kb_create: coerces (issue); kb_update: accepts non-ASCII-digit text and 5.5 (issue).
    • REST POST/PUT /api/entries: pydantic coerces true→1 and refuses 10.7 with a non-contract 422 (ran on an equivalent model).
    • Files on disk: top-level and metadata:-nested spellings (ran).
    • qa validate _check_importance_range (pyrite/services/qa_service.py) calls the same predicate on the index column, so tightening the predicate changes what qa reports for a REAL 5.5 in the index. Intended.
    • Plugins: extensions/journalism-investigation/.../validators.py has its own int-only range check. The model row fires first (test_a_journalism_kb_refuses_importance_99_with_the_models_answer), so it never disagrees. Leave it.

    Shapes (counted): 32 KINDS rows today (grep), × note/event × CLI/MCP × {file, set, create, add}. New kinds the decision names: "10", "1_0", "+5", " 5\n", !!binary NQ==, !!str 7. One new spelling axis: top-level vs metadata:-nested (on the set and file cells). REST create/update for true, 10.7, 5.5, "10". Types stay derived (TYPES: 11 core + 1 plugin + 1 kb.yaml-only).

    The issue's eight smaller limits, classified

    1. 5.5 accepted: in this theme (decision). Acceptance (2).
    2. Non-ASCII-digit text over MCP accepted and written as a string: in this theme (decision). Acceptance (3). The CLI half is the generic -f parser: separate issue (below).
    3. "The CLI cannot clear importance": already possible on dev. pyrite update <id> -k <kb> --unset importance removes the key (checked, ran; --unset landed with B6 P3b). No change; the row's "stated limit" becomes a pass that asserts --unset clears it.
    4. !!binary: in this theme, fixed by the same predicate (bytes are not a number). On dev only a binary that spells a number escapes (ran).
    5. REST 422 / coercion: in this theme for importance only, because REST turning true into 1 breaks the decision. Acceptance (6). The general "REST answers pydantic 422, not VALIDATION_FAILED, for every typed request field" is a separate issue; do not grow this theme.
    6. Kept row says model_validation: in this theme, one line in _validate_write. Acceptance (5).
    7. qa validate reads the index's coerced value: separate. It belongs to Name the one check for a hand-edited KB: five validators disagree and getting-started names none #1282 ("five validators disagree"), which is about whether qa reads files or the index. Tightening the predicate already makes qa flag a REAL 5.5.
    8. !!str 7 gets a row: in this theme. A YAML !!str scalar is text and is judged as its text ("7" is accepted on a write). Acceptance (4).

    4. Options

    • A. Keep the value as given in the model, and judge that (recommended). _base_kwargs, or build_entry for creates, records the raw importance when it is not exactly the int stored. validation_errors judges the raw value from both spellings, and validate_importance is tightened. Files: models/base.py, models/factory.py, schema/validators.py. Covers create, add, update and the nested spelling in one place. No in-flight PR touches these files.
    • B. Check fields["importance"] in KBService._prepare before build_entry. It is one site for creates, but it misses the nested spelling on update. It also edits the _prepare area that fix: a preset's type mapping applies where the KB declares its target (#1526) #1601 and fix(cli): create --template shows the fields a create writes in a KB (#1598) #1605 are changing. Not recommended.
    • C. Stop coercing: Entry.importance holds the raw value. About 20 readers of entry.importance / the column (grep: index, sort, csv, timeline, renderers) would need checking. Too wide for 0.26.4.

    5. Open questions

    • Whole floats (5.0, 10.0, 1e1). The decision refuses 5.5 and is silent on these. The current table says "good", and YAML 5.0 is a whole number. Recommended: accept a float whose value is whole (float.is_integer()). Note that a create of 5.0 then stores 5 and an update keeps 5.0 as sent, which is the same value. The maintainer may overrule at review; it is one row of the table.
    • Message. "Importance must be 1-10, got: 5.5" misleads. Recommended: keep rule: range and say "a whole number from 1 to 10" in message and expected on the base row. EventEntry's own message is pinned as unchanged by fix: a write refuses importance outside 1-10 on every type (#1537) #1570; check tests/test_plugin_contract.py, which pins the text.
    • Where the raw value lives. Entry is a dataclass and _base_kwargs returns constructor kwargs. A private non-field attribute set after construction, as capture_extra_frontmatter does for _unrepresented_keys, avoids touching _always_written_keys/_pristine_frontmatter. Prove that a no-op round trip writes no new bytes (tests/test_importance_range_on_every_type.py already compares the importance line).
    • Blank on create: CLI -f importance= is text "" and must be refused, as on update. MCP kb_create with null stays absent. The create then writes the default importance: 5, as every create does today. That default is a principle 3 question and is out of scope.

    6. Pointers

    pyrite/models/base.py (_base_kwargs, capture_extra_frontmatter, unrepresented_frontmatter, validation_errors), pyrite/schema/field_aliases.py read_view (how the nested spelling is lifted), pyrite/schema/validators.py validate_importance, pyrite/services/kb_service.py _validate_write, pyrite/server/schemas.py, tests/test_importance_range_on_every_type.py (KINDS, _surface_call), tests/test_rest_api.py (REST driver).

    7. Checked versus assumed

    Ran: 9 CLI creates; nested-spelling update and file; !!binary ×2, !!str 7, "7", 99 kept rows; --unset importance; --importance 999 on create and update (VALIDATION_FAILED, rule: range); pydantic coercion of 10 values. Read: the code named above. Assumed: the MCP rows (from the issue), REST route behaviour beyond the model, and whether a REST golden (tests/characterization/goldens/rest.*.json) moves when the request field's type changes.
    This groom is wrong if some create path builds an entry without _base_kwargs/build_entry (grep cls( in pyrite/formats/importers/ before starting; the CSV importer reads importance itself).

    Outside PR #1428 (typer min=1, max=10 on --importance) is now redundant. On dev, create --importance 999 and update --importance 0 are refused with VALIDATION_FAILED and the importance/range row (checked, ran). #1428 would put a second rule at the CLI with a usage error (exit 2) and a body MCP and REST do not give, which is against principle 8. Recommend closing it as superseded by #1570, with thanks and credit to the author. This theme does not touch the --importance option, so the two do not conflict in code.

    Theme 1588: one importance rule, judged on the value as given, on every write

    Acceptance:   (1) The KINDS table's create columns are replaced by "create verdict == set verdict" per surface, and the test asserts
                  that equality. Fails today: create -f importance=10.7 exits 0 and stores 10; true stores 1; high stores 5 (ran).
                  (2) 5.5, 10.5, 0.5 (and every non-whole float) are refused on create and update, and kept-and-reported on disk.
                  Fails today (5.5 accepted).
                  (3) MCP kb_update/kb_create with "10", "1_0", "+5", " 5\n", " 5 " are refused, file unchanged; "05" and "7" are
                  accepted. Fails today (written as strings).
                  (4) On disk: !!binary NQ== gets the kept warning (fails today); !!str 7 gets none (fails today); " 5 " gets the kept
                  warning (verdict changes from good to bad).
                  (5) A kept row carries the refusal's rule (importance: range) for every model row; the 3 tests pinning
                  model_validation are updated. Fails today.
                  (6) REST POST and PUT with importance true or 10.7: VALIDATION_FAILED with the importance row, nothing written, not
                  stored as 1 and not a FastAPI 422. Fails today.
                  (7) A file spelling metadata: {importance: 5}: update -f importance=10.7 is refused; a file holding
                  metadata: {importance: 10.7} gets the kept warning on a body update. Fails today (ran).
                  (8) update --unset importance clears the key (pass; replaces item 3's "stated limit").
                  One sentence in docs/json-contracts.md states the rule; changelog fragment names every answer that changes.
    Regimes:      on disk vs set vs create (ADR-0042 decision 8); null/blank = absent on disk and on MCP; "" text refused on CLI
                  create and update; event and non-event; journalism KB (plugin validator stays second).
    Shapes:       32 KINDS + 6 new kinds x note/event x CLI/MCP x {file, set, create, add}; x {top-level, metadata-nested} for set and
                  file; REST x 4 values. Verdicts by hand from the decisions.
    Touches:      existing: pyrite/schema/validators.py (~15), pyrite/models/base.py (~20), pyrite/models/factory.py (~5, if the raw
                  value is recorded there), pyrite/services/kb_service.py (_validate_write, 1 line), pyrite/server/schemas.py (2 fields),
                  docs/json-contracts.md (+2), tests/test_importance_range_on_every_type.py (~+120/-40), the 3 model_validation tests
                  (3 lines), tests/test_plugin_contract.py (if the message changes)
                  new: changelog.d/1588-*.changed.md
    Sequence:     kb_service.py: one line in _validate_write; #1601 (_prepare, _resolve_entry_type), #1605 (a method beside _prepare)
                  and #1576 (_templated_move, reconcile_templated_subdirectories) touch other functions, so either order works, with no
                  textual overlap expected. #1502/#1503 (casting list fields by declared type) queue behind #1601 in the same file but
                  in other functions. tests/test_importance_range_on_every_type.py is shared with #1584: land this first.
                  server/schemas.py: no PR in flight. If a REST golden moves, regenerate it after #1576, which edits rest.*.json.
    Model:        sonnet (the decisions are made, the table drives the work; the one judgement, where the raw value lives, is bounded)
    heavy:        no
    Cold read:    no (a value rule on one field; no storage, auth or schema format change). REST request schema change is small;
                  reviewer to confirm the OpenAPI shape is acceptable.
    Out of scope: the CLI -f parser reading "+5", "1_0", "05", " 5 " as ints for every field (pyrite/cli/entry_commands.py
                  _parse_field_value): not importance-specific; suggest a separate issue. REST pydantic 422 for other typed fields:
                  separate. qa reading the index (item 7): #1282. The create's always-written default importance: 5: principle 3,
                  separate. The journalism duplicate range check: safe (model fires first; pinned). --importance typer option: #1428,
                  see above.
    Decision:     none open. Whole floats (5.0) recommended accepted; the maintainer can overrule in review.
    

    One theme or two with #1584? Two. The invariants differ: one field's value rule here, the index's storability of every column and writer there. The code is disjoint except tests/test_importance_range_on_every_type.py. One worker can do both in sequence as two PRs, this one first. It has no decision pending and no cold read, which suits Thursday's cut.

  4. markramm commented on Nov 17, 2026

    @markramm
    CollaboratorAuthor

    Conductor (automated, for the maintainer), two notes on the groom above before anyone builds from it:

    1. Review: the groom says "cold read: no". This theme edits pyrite/server/schemas.py (what the REST create and update requests accept for importance), which is a public shape, so it gets a cold read and the maintainer merges it.
    2. One question is with the maintainer, not settled by the groom: whether a whole float (5.0, 10.0, 1e1) is accepted. The recorded decision says "importance is a whole number: 5.5 is refused" and does not name these. The groom recommends accepting a float whose value is whole. Until he answers, a worker builds the rest and leaves that row as the table has it today.

    🤖 Posted by the maintainer's conductor agent (Claude Code)

  5. markramm commented on Nov 17, 2026

    @markramm
    CollaboratorAuthor

    Claimed by the maintainer's loop, 2026-11-17: a cloud worker is building this from the groom above with the maintainer's decisions (a create refuses what an update refuses; whole numbers only; text that is not plain ASCII digits is refused), option A of the groom. The whole-float rows are left as they are until the maintainer answers. It posts its plan on a draft pull request before building; the draft will link here. Milestone 0.26.4. If you were about to start on it, say so here and the work can be shared or handed over.

    🤖 Written by the maintainer's conductor agent (Claude Code)

  6. markramm commented on Nov 17, 2026

    @markramm
    CollaboratorAuthor

    Claimed: draft pull request #1633.


    Generated by Claude Code

  7. markramm commented on Nov 17, 2026

    @markramm
    CollaboratorAuthor

    Decision (maintainer, 2026-11-17): a whole-number float (5.0) is accepted and stored as 5; 5.5 is refused, as decided.

    🤖 Recorded by the maintainer's conductor agent (Claude Code)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions