Skip to content

[10-1] Build corpus v2 with the expanded grammar #76

Description

@duckyquang

Goal 10, Task 1 — [10-1] · GEML Goals 6–12 (roadmap, README §5) · Size: L

Independence

Hard prerequisites (genuinely required): [10-0], [1-8]. These cannot be stubbed: it runs the merged v2 grammar through the merged Goal 1 generation/storage/QA pipeline.

What can start early, before the prerequisites merge: claim the issue, write the config and the tiny-fixture smoke test — only the work that touches real merged code or production artifacts must wait.

  • Exclusive write set — no other issue edits your owned paths, so your PR cannot conflict with parallel work.
  • Branch: issue/10-1-<short-slug>.

Objective

Generate corpus v2 with the expanded grammar under full 3R generator discipline: same determinism, dedup, split, and QA standards as the final v1 corpus.

Exclusive file ownership

Owned paths — this branch may edit only these paths:

  • src/geml/experiments/goal10/corpus_v2.py
  • configs/goal10_corpus_v2.yaml
  • docs/goals/goal10/CORPUS_V2_QA.md
  • tests/experiments/test_goal10_corpus_smoke.py

Read-only dependencies:

  • v2 grammar [10-0]
  • Goal 1 pipeline [1-4]–[1-8]

Tasks

  • Define v2 family quotas including trig families; document the delta from v1.
  • Generate deterministically with the [1-4]–[1-5] machinery; exact split counts; byte-reproducible.
  • Run the full [1-8]-style QA: uniqueness, quotas, depth mixture, triviality caps, domain validity.
  • Write shards/manifests under outputs/final/goal10/corpus_v2/.
  • Publish the corpus v2 QA report.

Acceptance criteria

  • Corpus v2 passes the same QA bar as v1 with exact split counts.
  • v1 corpus remains untouched and authoritative for v1 results.
  • Reproduction commands are documented.

Validation

  • python -m pytest tests/experiments/test_goal10_corpus_smoke.py
  • Run the production command in configs/goal10_corpus_v2.yaml.

Out of scope

  • Do not rerun compression studies; [10-2] owns that.

PR checklist

  • I edited only the owned paths listed above.
  • I did not inspect or reuse the previous GEML implementation.
  • Tests use temporary/tiny fixtures and do not require production outputs.
  • Failures and unsupported cases are retained and reported, not silently dropped.
  • I included exact reproduction/validation commands in the PR description.
  • I documented any scientific metric or assumption change.

Clean-room rule: Implement only from the current repository specifications and authoritative public sources. Use the v0 prototype only to understand what was measured and why — never as a code template. If an interface is ambiguous, stop and ask rather than modifying another issue's contract.

Metadata

Metadata

Assignees

Labels

clean-roomClean-room rule applies: no previous GEML implementationgoal:10Goal 10 — domain expansion (trig)group-aPipeline rebuild workstreamgroup-bArchitecture & learning workstreamneeds-review:dataRequires data reviewpriority:p0Critical pathtype:experimentProduction runs over real artifacts

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions