Define lightweight JSON parsing policy and harden crash-prone readers

[bearmug]

Summary

Pi-Rogue has many JSON parsing sites, but the fix should not be a noisy pass that wraps every JSON.parse call individually. The useful work is to define a small JSON parsing policy by data class, add lightweight shared helpers where they reduce boilerplate, and harden only the crash-prone/user-facing paths first.

Current signal

Malformed JSON or wrong-shape JSON can still cause problems in some flows:

• JSONL readers can abort an entire read on one bad line.
• Durable state/config files can silently fall back or crash depending on call site.
• TypeScript as SomeType assertions after JSON.parse provide no runtime validation.
• LLM-returned JSON sometimes needs best-effort parsing and graceful degradation.

This is real hardening signal, but it should not be treated as “wrap 29 parse calls” or “add schema validation everywhere.”

Proposed work

1. Classify JSON parse sites

Create a small inventory and policy for each class:

Class	Examples	Desired behavior
JSONL logs / ledgers	router ledgers, outcomes, labels	parse per-line; skip bad rows with warning/count; keep processing
User config	Pi-Rogue/router/context config	clear diagnostic; fallback only when safe
Durable session/state	goals, loop state, broker files	backup corrupt file when practical; fallback with warning
Model/training artifacts	binary gate, model cards	fail closed or ignore artifact with clear diagnostic
LLM-returned JSON	judge/router responses	best-effort parse; degrade gracefully
Trusted local/test data	fixed fixtures	can remain strict if failure should be loud

2. Introduce small helpers only where useful

Avoid one-off wrapping at every call site. Prefer helpers such as:

readJsonFile(path, fallback, options)
readJsonl(path, parseRow, options)
parseJsonObject(text, options)

The helper behavior should be explicit via options rather than magic global swallowing.

3. Patch highest-risk paths first

Prioritize flows that can affect normal Pi startup/session operation:

• session/router JSONL readers where one bad row aborts the whole read,
• durable state/config files that currently fail silently or crash unclearly,
• LLM JSON parsing paths where parse failure should degrade, not throw.

Non-goals

• Do not mechanically wrap every JSON.parse call.
• Do not add heavy schema validation everywhere.
• Do not turn trusted fixture/test parsing into noisy defensive code.
• Do not hide serious corruption without diagnostics.

Acceptance

• A short policy/inventory exists in code comments or docs near the helpers.
• Shared helpers cover at least JSON file and JSONL use cases.
• Highest-risk runtime readers use the helpers.
• Bad JSONL rows no longer abort an entire reader unless the caller explicitly requests strict mode.
• Corrupt durable files produce a useful warning and, where appropriate, a backup/fallback path.
• Tests cover representative behavior for strict, fallback, JSONL-skip, and corrupt-state cases.

Merged storage/state reader scope from #171

Also cover the storage-specific fallback problem originally tracked in #171:

• readText() / readJson() and orchestration readSessionJson() currently collapse missing files, unreadable files, and corrupt JSON into the same fallback behavior.
• Missing files are often expected and should remain quiet.
• Unexpected read failures or corrupt durable state should produce a diagnostic and, where practical, preserve the bad content with a backup before falling back.
• Write failures should continue to throw; this ticket is about read/fallback semantics and diagnostics, not hiding write errors.

This should be handled through the same lightweight policy/helper work above, not through a separate broad storage ticket.

[bearmug]

Mainline audit evidence for this open hardening issue, at aa595438650cfbaa6db310792929c6219cc003b2:

packages/advisor/src/extension.ts:448-471 calls .map/.slice on persisted collections without guarding their runtime types. A syntactically valid state file containing:

{"notes":{},"files":null,"errors":null}

reproduces session-start failure with (raw.notes ?? []).map is not a function. Invalid JSON and missing fields are tested, but valid JSON with wrong field types is not.

Suggested acceptance coverage for #167:

• normalize every persisted collection/scalar/object independently by runtime type and finite/range checks;
• preserve independently valid fields while defaulting malformed ones;
• cover wrong-type notes, files, errors, counters, router/check-in, and review-control fields;
• emit a bounded recovery diagnostic without failing session startup.