Skip to content

feat: record compiler version provenance - #521

Merged
smaramwbc merged 1 commit into
smaramwbc:mainfrom
MaximGit-tech:feat/compiler-version-metadata
Oct 14, 2026
Merged

smaramwbc merged 1 commit into
smaramwbc:mainfrom
MaximGit-tech:feat/compiler-version-metadata

Conversation

@MaximGit-tech

@MaximGit-tech MaximGit-tech commented Oct 13, 2026 •

Copy link
Copy Markdown

Summary

Adds compiler provenance metadata to every newly compiled memory so Statewave can tell which compiler configuration produced it.

This implements the missing compiler-version information requested in #443.

What changed

LLM compiler

LLM-produced memories now record:

  • compiler
  • model
  • prompt_version
  • compiler_version

prompt_version is a deterministic SHA-256 fingerprint of the prompt templates used to produce the memory.

The normal extraction pass versions the active _system_prompt(), including the compose-unit variant when enabled.

Recall-sweep memories are versioned separately. Their prompt version includes both the active main extraction prompt and _RECALL_SWEEP_PROMPT, because the sweep output also depends on the first-pass results it receives.

Recall-sweep memories continue to carry:

"pass": "recall_sweep"

Heuristic compiler

Heuristic-produced memories now record:

{
    "compiler": "heuristic",
    "compiler_version": "heuristic:1",
}

The heuristic compiler uses an explicit revision rather than hashing Python source code, so formatting or comment-only changes do not unnecessarily change the compiler version.

Existing claim metadata is preserved and merged with the new compiler provenance.

Structured candidates

Memories produced directly from structured memory candidates now record:

{
    "compiler": "structured",
    "compiler_version": "structured:1",
}

This keeps their provenance accurate because structured candidates bypass both LLM extraction and heuristic extraction.

Producer-supplied metadata is preserved, while the reserved compiler provenance fields are set by Statewave.

Shared versioning

Added a small compiler-version helper that produces deterministic SHA-256 fingerprints from version components.

Components are encoded separately before hashing so different component boundaries cannot accidentally produce the same fingerprint.

Backward compatibility

No database migration or backfill is required.

Existing memories without compiler_version remain unchanged and therefore continue to represent unknown compiler provenance, as described in #443.

Tests

Added regression coverage for:

  • LLM compiler provenance
  • compiler-version changes when the model changes
  • compiler-version changes when the active prompt changes
  • recall-sweep provenance
  • heuristic compiler provenance
  • preservation of heuristic claim metadata
  • structured candidate provenance
  • protection of reserved structured compiler metadata

Also updated existing heuristic claim tests to account for compiler provenance being present even when no claim is attached.

Validation:

1169 passed, 40 skipped, 3 warnings, 8 subtests passed
ruff check . -> All checks passed

The remaining warnings are pre-existing AsyncMock warnings unrelated to this change.

Closes #443

Signed-off-by: MaximGit-tech <maxim@MacBook-Air-Maksim.local>

@smaramwbc smaramwbc left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, this is thorough. All three write paths are covered, the tests bite, and no migration is needed since metadata_ is already JSONB. Merging, and I am following up with one tweak: the structured path reserves compiler and compiler_version but not model or prompt_version, so a candidate could still supply those two. Thanks!

@smaramwbc
smaramwbc merged commit 198e290 into smaramwbc:main Oct 14, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Record compiler type, model and prompt version on every compiled memory

2 participants