Skip to content

refactor: Beanie documents are the entities, with no domain copies or hand-written conversions - #424

Merged
HardMax71 merged 16 commits into
mainfrom
refactor/documents-as-entities
Oct 20, 2026
Merged

HardMax71 merged 16 commits into
mainfrom
refactor/documents-as-entities

Conversation

@HardMax71

@HardMax71 HardMax71 commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner

Summary

Closes #423, part of #308. Every stored entity lived three times, as a schema, a domain dataclass and a Beanie document, and routes and repositories copied between them with **model_dump() and asdict(). mypy can't check those copies, so #308's SecretStr change would have meant three unchecked edits per field.

Changes

The document is the entity now. Services and repositories take and return documents, the mirror dataclasses and their create and update variants are gone, and nothing under services/ or db/ imports a schema. An update's changes are a TypedDict that the route builds with a TypeAdapter: a field left out keeps its value, null clears a nullable one, and null on a field that can't be empty now answers 422. Embedded values are domain dataclasses in bson_encoders, and indexed fields are Annotated[..., Indexed()], so mypy sees their real types.

Warning

Where this departs from #423:

  • Unit tests run init_beanie once on a mocked database, so documents build through their constructors; model_construct() would skip validation.
  • SystemSettings stays. It's the value inside SystemSettingsDocument.config, not a copy of the document.
  • The settings update keeps null as "unchanged", since the settings page sends null for the sections it didn't touch.
  • The uncalled get_replay_status and ReplaySessionStatusInfo are deleted instead of converted.
  • mypy doesn't check document constructor arguments until lazy-model ships py.typed (BeanieODM/lazy_model#16).

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Refactoring (no functional changes)
  • CI/CD or tooling changes

Test Plan

  • Backend unit tests pass (cd backend && uv run pytest tests/unit)
  • Frontend unit tests pass (cd frontend && npm run test)
  • Backend E2E tests pass (docker compose exec -T backend sh -c 'uv sync --group test --no-dev --frozen --no-install-project --no-build && pytest tests/e2e')
  • Frontend E2E tests pass (cd frontend && npm run test:e2e)
  • API spec and client regenerated if routes or schemas changed (./deploy.sh openapi, then ./deploy.sh types), never edited by hand
  • Manual testing performed

Validation: ruff, mypy --strict, the orphan check, 573 unit tests, the Grafana contract tests and mkdocs build --strict pass. ./deploy.sh openapi leaves the spec unchanged. The full backend e2e suite with k3s in a container and the worker gives 493 passed, 1 skipped here and 489 passed, 1 skipped on main in the same setup. The 4 extra are the new tests.

Checklist

  • My code follows the project's style guidelines
  • I have run pre-commit run --all-files and all checks pass
  • I have added tests that prove my fix/feature works
  • I have updated documentation if needed
  • My changes don't introduce new warnings

Summary by cubic

Makes Beanie documents the single entity for stored data. They previously lived three times — as an API schema, a domain dataclass, and a Beanie document — with routes and repositories copying between them via **model_dump() and asdict(); those copies bypass mypy, so the planned SecretStr change would have meant three unchecked edits per field. Services and repositories now take and return documents, the mirror dataclasses and their create/update variants are gone, and nothing under services/ or db/ imports a schema.

Behavior and adoption notes

  • Update changes are TypedDicts validated by TypeAdapter in the routes: a missing key keeps the stored value, null clears a nullable field, and null on a field that can't be empty answers 422.
  • The user settings endpoint keeps null as "unchanged", since the settings page sends null for sections it didn't touch.
  • SystemSettings stays as the value inside SystemSettingsDocument.config, and the uncalled get_replay_status and ReplaySessionStatusInfo are deleted instead of converted.
  • Indexed fields are declared Annotated[..., Indexed()] so mypy sees the wrapped type; document constructor arguments aren't checked until lazy-model ships py.typed.
  • Unit tests run init_beanie once on a mocked database, so documents build through their validated constructors rather than model_construct().
  • The OpenAPI spec is unchanged; repository and service signatures changed, so callers now build documents directly.

Written for commit 02c38fe. Summary will update on new commits.

View guided diff Turn on auto-fix

…eclare indexes as Annotated so mypy sees their types; creates insert the caller's document
…ageChanges for status updates; RetryPolicy takes the retry count
…th ReplaySessionChanges for updates; the document embeds the domain ReplayConfig instead of a copy, and the uncalled get_replay_status goes
…UserSettingsChanges for updates; events apply field by field instead of through dict merges
…ead of splatting a dump into the constructor
…the same; a comment says why null keeps a value
…nd update changes, schemas stay in the routes
@HardMax71
HardMax71 enabled auto-merge (squash) October 20, 2026 18:43
@coderabbitai

coderabbitai Bot commented Oct 20, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 77f72aa8-3e6e-45e6-8135-f34cafa89f37
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added documentation Improvements or additions to documentation area: api FastAPI routes, auth, rate limiting, settings area: execution Queue, scheduler, k8s worker, pod monitor, result processor, executor pods, runtimes area: events Kafka consumers and middlewares, producer, idempotency, DLQ, replay area: realtime SSE streams and notifications labels Oct 20, 2026
@sonarqubecloud

Copy link
Copy Markdown

@codecov-commenter

codecov-commenter commented Oct 20, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 96.35535% with 16 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
backend/app/api/routes/admin/users.py 77.77% 2 Missing ⚠️
backend/app/api/routes/notifications.py 81.81% 2 Missing ⚠️
backend/app/db/repositories/replay_repository.py 75.00% 2 Missing ⚠️
backend/app/db/repositories/saga_repository.py 89.47% 2 Missing ⚠️
backend/app/services/user_settings_service.py 94.59% 2 Missing ⚠️
backend/app/db/repositories/dlq_repository.py 88.88% 1 Missing ⚠️
.../db/repositories/resource_allocation_repository.py 50.00% 1 Missing ⚠️
...nd/app/db/repositories/user_settings_repository.py 88.88% 1 Missing ⚠️
backend/app/services/admin/admin_user_service.py 94.11% 1 Missing ⚠️
backend/app/services/notification_service.py 95.23% 1 Missing ⚠️
... and 1 more
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Flag Coverage Δ
backend-e2e 82.25% <94.30%> (-0.31%) ⬇️
backend-unit 74.73% <72.66%> (-0.14%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
backend/app/api/dependencies.py 100.00% <100.00%> (ø)
backend/app/api/routes/admin/events.py 100.00% <100.00%> (ø)
backend/app/api/routes/admin/executions.py 100.00% <100.00%> (ø)
backend/app/api/routes/admin/rate_limits.py 93.10% <100.00%> (ø)
backend/app/api/routes/admin/settings.py 100.00% <100.00%> (ø)
backend/app/api/routes/execution.py 100.00% <100.00%> (ø)
backend/app/api/routes/replay.py 100.00% <100.00%> (ø)
backend/app/api/routes/saga.py 100.00% <100.00%> (ø)
backend/app/api/routes/saved_scripts.py 100.00% <100.00%> (ø)
backend/app/api/routes/sse.py 100.00% <100.00%> (ø)
... and 63 more
Components Coverage Δ
Backend 92.59% <96.35%> (+0.13%) ⬆️
Frontend 92.27% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@HardMax71
HardMax71 merged commit fd04edf into main Oct 20, 2026
22 checks passed
@HardMax71
HardMax71 deleted the refactor/documents-as-entities branch October 20, 2026 18:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: api FastAPI routes, auth, rate limiting, settings area: events Kafka consumers and middlewares, producer, idempotency, DLQ, replay area: execution Queue, scheduler, k8s worker, pod monitor, result processor, executor pods, runtimes area: realtime SSE streams and notifications documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refactor: Beanie documents are the entities, with no domain copies or hand-written conversions

2 participants