Added compression competition mode

.gitignore

@@ -1,8 +1,13 @@
 __pycache__/
+**/__pycache__/
+*.py[cod]
+.pytest_cache/
+.ruff_cache/
+competition_artifacts/
+miner/.competition-sdk/
 note.txt
 *.log
 *.db
-*.mp4
 *.lock
 *.hevc
 *.y4m
@@ -12,8 +17,7 @@ note.txt
 *.js
 *.xml 
 .DS_Store
-*.json
-.env
+*.env
 venv/
 video2x/
 logs/

competition_solution.json

@@ -0,0 +1,9 @@
+{
+  "competition_type": "COMPRESSION",
+  "entrypoint": "miner/modal_workers.py",
+  "local_path_io": true,
+  "preflight": "miner/common_preflight.py",
+  "routes": ["/health", "/compress"],
+  "schema_version": 2,
+  "sdk": "miner/competition_sdk.py"
+}

competitions/fixtures/README.md

@@ -0,0 +1,18 @@
+# Compression warmup fixture
+
+`compression_warmup_input.mp4` is a five-second, 320×180 synthetic test pattern
+generated with FFmpeg's `testsrc2` source. It contains no third-party footage or
+audio and may be redistributed with this repository.
+
+Regenerate it with:
+
+```bash
+ffmpeg -f lavfi -i "testsrc2=size=320x180:rate=24:duration=5" \
+  -an -c:v libx264 -pix_fmt yuv420p -movflags +faststart \
+  competitions/fixtures/compression_warmup_input.mp4
+```
+
+The competition preflight validates that the fixture exists, contains a video
+stream, is no longer than 5.5 seconds, and is no larger than 720p. The unscored
+warmup requires `/compress` to preserve dimensions and timing while emitting a
+smaller AV1 MP4.

competitions/manifests/examples/compression-competition.json

@@ -0,0 +1,46 @@
+{
+  "schema_version": 2,
+  "scoring_version": "3",
+  "competition_id": "compression-2026-w29",
+  "competition_type": "COMPRESSION",
+  "competition_start_time": "2026-07-16T00:00:00Z",
+  "contender_ping_interval": "30m",
+  "contender_finalisation_time": "2026-07-17T00:00:00Z",
+  "human_review_deadline": "2026-07-22T23:30:00Z",
+  "competition_end_time": "2026-07-23T00:00:00Z",
+  "required_routes": ["/compress"],
+  "allowed_gpus": ["L4", "L40S", "RTX-PRO-6000"],
+  "max_cpu_cores": 32,
+  "requested_cpu_cores": 16,
+  "container_size_limit_gb": 25,
+  "modal_build_timeout": "20m",
+  "evaluation_batch_size": 5,
+  "evaluation_batched_run_timeout": "10m",
+  "scoring_batched_run_timeout": "5m",
+  "min_video_length": "5s",
+  "max_video_length": "1h",
+  "length_weight_exponent": 2.02,
+  "required_output_codec": "AV1",
+  "vmaf_threshold": 90.0,
+  "vmaf_sample_count": 10,
+  "minimum_compression_ratio": 1.25,
+  "scoring_seed": 20260716,
+  "warmup_input_path": "competitions/fixtures/compression_warmup_input.mp4",
+  "boss": {
+    "repository_path": null,
+    "boss_hotkey": null
+  },
+  "evaluation_input_volume_name": "vidaio-competition-compression-2026-w29-inputs",
+  "evaluation_index_path": "/validator-evaluation/index.json",
+  "output_volume_prefix": "vidaio-competition-compression-2026-w29-output",
+  "max_parallel_contenders": 4,
+  "max_attempts_per_item": 2,
+  "scoring_factors": {
+    "quality": "0.6",
+    "cost_efficiency": "0.25",
+    "length_coverage": "0.15",
+    "runtime": "0"
+  },
+  "cost_floor_usd": "0.000001",
+  "score_precision": 8
+}

docs/competition_phase0_feasibility_report.md

@@ -0,0 +1,58 @@
+# Competition mode Phase 0 feasibility report
+
+Date: 2026-07-14
+Scope: compression competition mode only
+Decision: **Phase 0 is implemented but not cleared for production.**
+
+## Result summary
+
+| Gate | Result | Evidence and remaining work |
+|---|---|---|
+| 25 GB hostile-build limit | Partial | The exact 25,000,000,000-byte boundary accepts 25 GB and rejects 25 GB plus one byte. A POSIX hard file-size limit stopped a hostile writer at its configured boundary. The local Docker daemon was unavailable, and neither Docker's external daemon nor Modal's image builder inherits this process limit, so an end-to-end trusted-builder proof is still required. |
+| Offline Modal runtime | Passed | A live dev Sandbox blocked direct-IP and HTTPS egress. No Modal account credential or OIDC identity token was present inside it. The CPU-only probe used bounded CPU/memory, no attached secrets or GPU, and a 60-second timeout; afterward the app was stopped with zero tasks. |
+| Raw PAT boundary | Partial | The raw PAT is present only in the wire DTO. Representative application, Bittensor debug, PM2, Git environment, Redis, SQLite, W&B, and exception values are redacted or use a PAT-free persistent DTO. The test must be repeated through the production Bittensor submission handler and real configured log/export sinks when those Phase 1 components exist. |
+| Billing granularity and allocation | Passed | Live workspace access returned a valid hourly report (zero rows were expected for the newly created dev environment). The API groups by Modal object, resource, and requested tags rather than native evaluation-item cost. The implemented allocation assigns a batch bill by active-runtime share, uses an equal share if every runtime is zero, and preserves the exact billed `Decimal` total. |
+| Competition formula | Passed | Tests prove 60% length-weighted compression effectiveness, 25% per-item contender-relative cost efficiency, and 15% length-weighted completion coverage. The cheapest valid result for an item receives cost efficiency 1.0; failed and invalid results receive zero and cannot define that minimum. The Phase 4 scorer supplies zero effectiveness below the VMAF floor. Longer inputs carry greater logarithmic weight, terminal failures contribute zero, and every contender must have one terminal row for every manifest input. |
+
+The dependency-free local runner still reports its live-only gate as blocked by design. The separate machine-readable live result is in `docs/competition_phase0_modal_report.json` and passes both network isolation and billing access. Overall production readiness remains false because the trusted hostile-builder proof and actual protocol/sink PAT canary are not complete.
+
+## Implemented artifacts
+
+- `vidaio_subnet_core/competition/phase0.py`: policy gates, redaction boundary, approved scoring formula, and bill allocation.
+- `scripts/competition_phase0.py`: dependency-free local runner and JSON report generator.
+- `scripts/competition_modal_phase0_probe.py`: bounded live Modal network-isolation and billing capability probe.
+- `tests/competition/test_phase0.py`: local reproducible contract tests.
+- `docs/competition_phase0_modal_report.json`: sanitized live Modal evidence.
+
+## Reproduce the local evidence
+
+```bash
+PYTHONDONTWRITEBYTECODE=1 python3 -m unittest tests.competition.test_phase0 -v
+PYTHONDONTWRITEBYTECODE=1 python3 scripts/competition_phase0.py
+```
+
+On the recorded workstation, all 12 unit tests passed. The local gate runner proved the formula and local primitives and reported partial results for image enforcement and PAT integration. The live Modal gate was then executed separately and passed.
+
+## Run the live Modal gate
+
+Do not paste the token into logs, source files, CLI arguments, or this report. Supply a dev-workspace token through the process environment and run:
+
+```bash
+export MODAL_TOKEN_ID='...'
+export MODAL_TOKEN_SECRET='...'
+env UV_CACHE_DIR=/private/tmp/vidaio-modal-phase0-uvcache \
+  uv run --python 3.12 --with 'modal>=1.4.0' \
+  python scripts/competition_modal_phase0_probe.py --environment dev
+unset MODAL_TOKEN_ID MODAL_TOKEN_SECRET
+```
+
+The probe creates one short-lived CPU-only Sandbox with no attached secrets or GPU. It attempts direct-IP and HTTPS egress, verifies account credentials and an OIDC identity token are absent inside the Sandbox, and then requests the last two complete hourly billing intervals. Billing reports may require a Team or Enterprise workspace. If the workspace has no `dev` environment, create it first or explicitly select the intended non-production environment.
+
+The probe was API-checked and executed against Modal SDK 1.5.2 on 2026-07-14. A dedicated `dev` environment was created because the workspace initially contained only `main`. The runtime network probe was repeated after Phase 3 with explicit DNS, direct-IP, and HTTPS attempts; all three failed while Modal account credentials remained absent. Relevant current Modal documentation: [network controls](https://modal.com/docs/guide/sandbox-networking), [resource limits](https://modal.com/docs/guide/sandbox-resources), [workspace billing reports](https://modal.com/docs/sdk/py/latest/modal.Workspace), and [token environment variables](https://modal.com/docs/sdk/py/latest/modal.config).
+
+## Production blockers
+
+1. Select and prove a trusted build mechanism that enforces the 25 GB limit while layers are being created. Post-build inspection is not sufficient because it does not cap hostile build-time disk or spend. Modal's public Image API does not expose a dependable arbitrary image-size quota/measurement for this gate.
+2. Repeat the PAT canary through the real Bittensor submission handler and configured PM2, Git, Redis, SQLite, W&B, and exception paths after Phase 1 creates that handler.
+
+Phase 1 contract work may continue behind the default-off feature flag, but no contender build or production competition should run until the trusted-builder gate is cleared. The PAT integration repeat is a mandatory pre-production gate once the submission path exists.

docs/competition_phase4_dataset_runbook.md

@@ -0,0 +1,92 @@
+# Competition Phase 4 dataset runbook
+
+Phase 4 does not dispatch until the evaluation input Volume contains a verified
+index and that exact index digest is sealed in the competition SQLite database.
+Creating an empty Modal Volume is therefore only the first step.
+
+Set these shell variables to the manifest the validator will load, the directory
+containing the private source MP4 files, and a local generated index path:
+
+```bash
+MANIFEST=/absolute/path/to/compression-2026-w30.json
+SOURCE_DIR=/absolute/path/to/private-evaluation-videos
+INDEX=/tmp/compression-2026-w30-index.json
+DATABASE_URL=sqlite:////absolute/path/to/video_subnet_validator.db
+```
+
+Prepare and locally verify the immutable index:
+
+```bash
+python scripts/competition_dataset.py prepare \
+  --manifest "$MANIFEST" \
+  --source-dir "$SOURCE_DIR" \
+  --index "$INDEX"
+
+python scripts/competition_dataset.py validate \
+  --manifest "$MANIFEST" \
+  --source-dir "$SOURCE_DIR" \
+  --index "$INDEX"
+```
+
+For compression competitions, `prepare` creates one evaluation query per
+physical source MP4. It deterministically selects CRF or VBR and a VMAF floor
+of 85, 89, or 93 using the manifest's `scoring_seed`; VBR queries also receive
+a deterministic random target of 5, 8, or 10 Mbps. Consequently, 20 source
+videos produce 20 evaluation rows. The format remains evaluation index schema
+version 2; validators continue to read older indexes for sealed competitions.
+
+Upload to the manifest's `evaluation_input_volume_name` in Modal `main`. The
+command verifies the uploaded index and every source object by reading them back.
+It refuses to replace a different index already present in the Volume.
+
+```bash
+python scripts/competition_dataset.py upload \
+  --manifest "$MANIFEST" \
+  --source-dir "$SOURCE_DIR" \
+  --index "$INDEX" \
+  --environment main
+```
+
+Seal the same digest in the validator database only after upload succeeds:
+
+```bash
+python scripts/competition_dataset.py seal \
+  --manifest "$MANIFEST" \
+  --index "$INDEX" \
+  --environment main \
+  --database-url "$DATABASE_URL"
+```
+
+The validator does not need to be running. For a new database, `seal` applies
+the competition schema baseline, registers the manifest as `SCHEDULED`, and
+seals the evaluation rows. On first boot, configure `COMPETITION_DATABASE_URL`
+with this same URL and ensure the manifest remains matched by
+`COMPETITION_MANIFEST_GLOB`; the competition will start automatically when its
+start time is due.
+
+The late-loading workflow remains supported. If the validator was started with
+an empty input Volume, run `upload` and `seal` against its existing database.
+The manifest digest must match, and an `EVALUATING` competition resumes on its
+next execution cycle.
+
+A sealed index is immutable. Do not try to apply the one-query-per-video format
+to an already sealed competition; prepare a fresh index and competition ID.
+Miners must publish from the matching SDK revision because VBR requests add
+`codec_mode` and `target_bitrate` to the competition route contract.
+
+On the next scheduler cycle, every accepted contender receives sequential
+batches of at most five distinct source videos. Different contenders run
+concurrently up to the manifest's `max_parallel_contenders`. A restart resumes
+from SQLite. An expired in-flight batch is marked terminally failed and is not
+automatically dispatched again. A confirmed validator-infrastructure incident
+can only be replayed through the explicit, audited operator repair command.
+
+Expected logs include `Dispatching competition batch`, `Competition batch
+persisted`, both `EVALUATING -> SCORING` and `SCORING -> AWAITING_END_TIME`, and
+finally `Competition Phase 4 complete`. Each contender Sandbox is terminated as
+soon as that hotkey finishes every dataset item, without waiting for slower
+contenders, and logs `trigger=contender_dataset_complete` plus
+`volume_retained=true`. Input and contender output Volumes and their files
+persist until manually deleted. A checksum mismatch, missing index, or missing
+output remains in `EVALUATING` with a reason-coded log instead of silently
+scoring incomplete data.