You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Builds on #50 (as-of reads). Part of the point-in-time set with #51 and the retraction issue.
Why a diff here is stronger than a text diff
Content-addressing makes a difference attributable, not merely visible:
observation
meaning
same input_key, different output_key
identical recipe — the model reached a different conclusion
different input_key
the recipe changed; compare its components to say which
And the components are already separated: analyzer version, config fingerprint, source-set hash. So a diff can state precisely why two nodes differ — the author bumped the analyzer, the user changed config or model, or the inputs moved. A text diff cannot distinguish those, and that distinction is usually the whole question.
Proposed change
prospect diff --unit <analyzer> <source_set_hash> # across the revises chain
prospect diff --runs <A> <B> # what two runs concluded differently
prospect diff --as-of <T1> <T2> # added / removed / changed by analyzer
Node content is canonical JSON, so comparison should be structural (changed fields) rather than line-based, with a summary line per node and details on request.
Motivating uses, all encountered in practice
"How did our conclusion about this turn change when the prompt improved?" The revises chain already links versions of one logical unit; nothing surfaces the content delta. Answering it currently means reading content_json by hand.
Model and prompt bake-offs. Because the resolved model is part of the config fingerprint, different models' verdicts on the same term already coexist as separate nodes. Comparing them is the natural read — and it was recently done by hand-writing a throwaway SQL harness, when --runs A B would have been a first-class command. The same applies to comparing prompt versions: v1.1 vs v1.2 verdicts sit side by side already.
Regression triage. When proposal quality moves, --as-of localises when, and --unit shows what actually changed in the conclusion.
Notes
--runs needs run ids to be discoverable; prospect analyzers or a prospect runs listing should show recent runs with their mode, model spec, and node counts.
For analyzers with many nodes per session, default to a summary (counts of added / removed / changed per analyzer) and require a selector for full content diffs.
Builds on #50 (as-of reads). Part of the point-in-time set with #51 and the retraction issue.
Why a diff here is stronger than a text diff
Content-addressing makes a difference attributable, not merely visible:
input_key, differentoutput_keyinput_keyAnd the components are already separated: analyzer version, config fingerprint, source-set hash. So a diff can state precisely why two nodes differ — the author bumped the analyzer, the user changed config or model, or the inputs moved. A text diff cannot distinguish those, and that distinction is usually the whole question.
Proposed change
Node content is canonical JSON, so comparison should be structural (changed fields) rather than line-based, with a summary line per node and details on request.
Motivating uses, all encountered in practice
reviseschain already links versions of one logical unit; nothing surfaces the content delta. Answering it currently means readingcontent_jsonby hand.--runs A Bwould have been a first-class command. The same applies to comparing prompt versions: v1.1 vs v1.2 verdicts sit side by side already.--as-oflocalises when, and--unitshows what actually changed in the conclusion.Notes
--runsneeds run ids to be discoverable;prospect analyzersor aprospect runslisting should show recent runs with their mode, model spec, and node counts.