Skip to content

fix(consolidation): default invalid dedup actions to keep#2565

Merged
benfrank241 merged 1 commit into
vectorize-io:mainfrom
r266-tech:r266/dedup-action-keep-fallback
Jul 7, 2026
Merged

fix(consolidation): default invalid dedup actions to keep#2565
benfrank241 merged 1 commit into
vectorize-io:mainfrom
r266-tech:r266/dedup-action-keep-fallback

Conversation

@r266-tech

Copy link
Copy Markdown
Contributor

Summary

Fixes #2545.

Consolidation dedup already defaulted a missing structured action to keep, but near-miss or malformed action values from a lax structured-output model could still fail Pydantic validation before the conservative no-merge path ran.

This adds a mode="before" validator on _DedupDecision.action that:

  • preserves exact "merge" / "keep" decisions
  • logs invalid action values
  • defaults every other scalar or non-scalar shape to "keep"

That keeps malformed dedup verdicts from accidentally merging observations or requeueing the consolidation job.

Tests

  • ruff check hindsight_api/engine/consolidation/consolidator.py tests/test_consolidation_dedup.py
  • ruff format --check hindsight_api/engine/consolidation/consolidator.py tests/test_consolidation_dedup.py
  • python3 -m py_compile hindsight_api/engine/consolidation/consolidator.py tests/test_consolidation_dedup.py
  • local validator smoke check for exact valid actions, unknown string actions, near-miss Merge, list/dict values, and missing action

I could not run pytest tests/test_consolidation_dedup.py in this macOS arm64 environment: uv run resolves onnxruntime==1.27.0, which has no wheel for the current Python/platform combo, and direct PYTHONPATH=. pytest ... -o addopts='' cannot collect without the project runtime deps such as asyncpg installed.

Codex review noted a low-confidence import concern about field_validator; the module already imports it from Pydantic and py_compile passed.

@benfrank241

Copy link
Copy Markdown
Member

lgtm. defaulting anything that isn't exactly merge/keep to keep is the right conservative call. Merging. Thanks!

@benfrank241 benfrank241 merged commit 0cde79b into vectorize-io:main Jul 7, 2026
87 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

consolidation: dedup verdict rejects near-miss action values from lax-JSON models — _DedupDecision fails validation instead of degrading to keep

2 participants