Skip to content

Quarantine private Course Resources and verify downloads - #318

Draft
Vinosaamaa wants to merge 10 commits into
mainfrom
codex/244-private-storage
Draft

Vinosaamaa wants to merge 10 commits into
mainfrom
codex/244-private-storage

Conversation

@Vinosaamaa

@Vinosaamaa Vinosaamaa commented Sep 12, 2026

Copy link
Copy Markdown
Owner

Summary

Course Resource uploads now remain private and quarantined until validated bytes pass a real ClamAV scan. The media module owns immutable object keys, upload idempotency, durable recovery/deletion, verified authorized downloads, and conservative byte/request accounting for a free S3-compatible provider.

Refs #244. Depends on #243 for production configuration, schema epoch 4 and measured capacity. GitHub Actions and independent root review passed at 2042bd4; external CodeAnt then identified the plaintext scanner transport. The Unix socket follow-up is locally verified and awaits its own native, product and security gates. Keep this draft for the pending #319/#243 integration and epoch 4 rollout. The owning issue stays open through actual private-provider and browser/deployment proof; no external account or resource is provisioned by this PR.

Acceptance Criteria

  • Private configurable S3 storage (planned OCI Always Free; compatible alternatives and explicit local adapter), with no public object URLs or paid fallback.
  • Actual byte/type/filename/checksum validation and bounded transfers; ownership-checked attachment downloads verify the stored checksum before returning content.
  • Flyway V2 durable quarantine, scan leases, idempotency across failure/deletion, replay-safe completion, confirmed cleanup and conservative quota release.
  • 202 upload response with public status/checksum, instructor deletion/usage, learner visibility only after clean scan; no internal storage/scanner details in responses.
  • Real ClamAV over a private Unix socket with fresh definitions and bounded read/write deadlines; failed scans never publish. Legacy migration clears old chunks and rescans while retaining original files.
  • Attempt accounting before every S3 call, SDK retries disabled, protected deletion headroom and prefix/age-constrained reconciliation.
  • Persisted storage namespace binding blocks accidental bucket/backend changes; reviewed copy and checksum inventory is required for migration.

Test Plan

  • Java 21 affected-module Maven verify: 39 local media tests, including concurrent idempotency/request caps, stale leases, deletion races, corrupt downloads, uncertain writes, scanner failures and migration recovery.
  • Additional regressions prove clean-file preservation through indexing outages and startup rejection of a changed storage namespace before network access.
  • Full CI, including all 14 real product browser journeys, passed at 2042bd4.
  • TDD failures reproduced for blocked scanner writes and provider-stream closure when spool creation fails; corrected tests pass.
  • Compose configuration and Actionlint validation; product scanner wiring, safe seed filename and AVAILABLE/index-chunk polling.
  • Native AMD64/ARM64 PostgreSQL, S3Mock and real ClamAV integration passed at 2042bd4, including EICAR/scan-limit rejection and restart durability. Actual process run.
  • Actual private OCI policy/anonymous denial, full 2 OCPU/12 GB workload and browser processing/failure journeys remain deployment acceptance.

Architecture And Operations Notes

  • Service boundaries changed: media owns the lifecycle and adapters; existing resource chunk routes are reused.
  • Database migration: media V2. Older code ignores quarantine, so deployment must advance schema epoch from 3 to 4 and disallow old-code rollback.
  • API contract: upload 202 plus status/sha256; additive metadata/deletion/course-usage routes. Existing access-token/course authorization remains authoritative.
  • Limits: 8 GB resources; 40,000 monthly object attempts, including a 4,000 DELETE reserve. Independent backups/operators must fit the remaining free account allowances.
  • Scanner startup completes a foreground signature update before clamd loads its engine; native diagnostics reproduced stale loaded definitions after the earlier concurrent startup.
  • Scanner transport: Unix-only daemon, directory mode 2770 and socket mode 0660 shared only with the media process. The optional client TCP mode defaults off, accepts only loopback and never acts as a fallback.
  • Scanner capacity: 4 GB container design target; cannot be added to the prior 7.625 GiB base without reallocating and measuring. No production capacity claim.
  • Legacy resources and AI retrieval remain under maintenance through purge/import/scan. Original files are retained for reviewed recovery.
  • Logs expose resource IDs and generic operation phases, not provider responses or scanner signatures.

Engineering impact

  • None
  • Change Note
  • ADR
  • Architecture Review
  • Feature Retrospective
  • Postmortem
  • Capability Dossier

Exact record: architecture-review-chanter-private-resources@1.

Checklist

  • Scope is one private-resource lifecycle and operator recovery path.
  • Media checks course permissions before protected operations.
  • No production secrets, machine-local identifiers or private endpoints are committed.
  • System review and migration/recovery runbook distinguish implementation from outstanding deployment proof.
  • Numbered Engineering receipt and exact rich-record revision are committed.

@codeant-ai

codeant-ai Bot commented Sep 12, 2026

Copy link
Copy Markdown

🏁 CodeAnt Quality Gate Results

Commit: 01c5b442
Scan Time: 2026-09-12 06:31:40 UTC

✅ Overall Status: PASSED

Quality Gate Details

Quality Gate Status Details
Duplicate Code ✅ PASSED 0.1% duplicated
SAST ✅ PASSED No security issues
IAC ✅ PASSED Rating S: No issues

View Full Results

@Vinosaamaa
Vinosaamaa force-pushed the codex/244-private-storage branch from 3fa4bdb to b3e9da8 Compare September 12, 2026 05:34
@Vinosaamaa Vinosaamaa changed the title feat(media): quarantine private Course Resources and verify downloads Quarantine private Course Resources and verify downloads Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant