Skip to content

Validate sample and rubric JSON with Pydantic - #106

Open
JeremyZeng77 wants to merge 1 commit into
mergeos-bounties:masterfrom
JeremyZeng77:codex/nokaman-pydantic-validation-4
Open

Validate sample and rubric JSON with Pydantic#106
JeremyZeng77 wants to merge 1 commit into
mergeos-bounties:masterfrom
JeremyZeng77:codex/nokaman-pydantic-validation-4

Conversation

@JeremyZeng77

Copy link
Copy Markdown

Summary

  • Adds Pydantic validation models for sample and rubric JSON payloads.
  • Keeps load_sample() and load_rubric() returning dictionaries so existing CLI/GUI/eval callers keep working.
  • Reports invalid payloads with source path and field path details for easier fixture debugging.
  • Adds tests that validate all existing sample/rubric fixtures and assert helpful errors for invalid payloads.

Fixes #4.

Validation

  • python -m pytest tests/test_data_schema_validation.py tests/test_eval.py tests/test_language_coverage.py -q (11 passed)
  • python -m ruff check src tests (passed)
  • python -m compileall src tests (passed)
  • python -m pytest -q (71 passed)
  • git diff --check (passed)

Bounty

Claimed in #4 and MergeOS Claim Token #1 with required repos starred.

@JeremyZeng77

Copy link
Copy Markdown
Author

Light follow-up: this PR is still ready for review. It keeps the existing loader dict interface intact and includes validation evidence in the PR body (
uff, compileall, and full pytest -q passing locally). Happy to adjust if maintainers prefer a different schema strictness level.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[25 MRG] Rubric schema validation with pydantic

1 participant