[11-0] Build corpus v3 and equivalence pairs at 10-100x scale

[duckyquang]

Goal 11, Task 0 — [11-0] · GEML Goals 6–12 (roadmap, README §5) · Size: L

Independence

Hard prerequisites (genuinely required): [10-1], [6-1]. These cannot be stubbed: it scales the merged corpus and pair pipelines to 10–100x — throughput engineering over real, merged machinery.

What can start early, before the prerequisites merge: claim the issue, write the config and the tiny-fixture smoke test — only the work that touches real merged code or production artifacts must wait.

• Exclusive write set — no other issue edits your owned paths, so your PR cannot conflict with parallel work.
• Branch: issue/11-0-<short-slug>.

Objective

Scale the corpus and equivalence-pair pipelines to 10–100x (corpus v3 + pairs) with unchanged quality bars: determinism, dedup, splits, leakage checks, and QA at scale.

Exclusive file ownership

Owned paths — this branch may edit only these paths:

• src/geml/experiments/goal11/corpus_v3.py
• configs/goal11_corpus_v3.yaml
• tests/experiments/test_goal11_corpus_smoke.py

Read-only dependencies:

• Corpus pipeline [1-4]–[1-8]/[10-1]
• pair pipeline [6-1]

Tasks

• Profile and remove throughput bottlenecks (parallel shard generation, streaming dedup) without changing outputs.
• Generate corpus v3 and the scaled pair set with exact split counts and manifests.
• Re-verify determinism and leakage at scale (sampled byte-reproducibility checks).
• Record throughput, memory, and cost telemetry for the scaling report.
• Write under outputs/final/goal11/ with full manifests.

Acceptance criteria

• Quality bars are identical to v1/v2 (same QA suite passes).
• Scale telemetry is recorded for [11-1].
• No quality check was weakened to hit scale.

Validation

• python -m pytest tests/experiments/test_goal11_corpus_smoke.py
• Run the production command in configs/goal11_corpus_v3.yaml.

Out of scope

• Do not train models or produce scaling curves.

PR checklist

• I edited only the owned paths listed above.
• I did not inspect or reuse the previous GEML implementation.
• Tests use temporary/tiny fixtures and do not require production outputs.
• Failures and unsupported cases are retained and reported, not silently dropped.
• I included exact reproduction/validation commands in the PR description.
• I documented any scientific metric or assumption change.

Clean-room rule: Implement only from the current repository specifications and authoritative public sources. Use the v0 prototype only to understand what was measured and why — never as a code template. If an interface is ambiguous, stop and ask rather than modifying another issue's contract.