Repository navigation
Deliver real RTPTorrent Parquet ETL and native replay #302
Description
Parent: #261. Milestone: M5 — Generalizable TCPCI benchmark.
Objective and bounded deliverable
Build a complete installed-package real-data workflow using the reviewed source selection profile and existing RTPTorrent reader. Reuse the existing cache, artifact writer, native API, sinks/checkpoints and versioned results.
- Freeze a bounded public raw-class slice with exact source/member SHA256, byte counts, component terms and citations; preserve all job/class/position identities and raw outcome counters.
- Emit deterministic analytical and native-compatible Parquet, conserved exclusions and a completion/provenance receipt written last. Unknown/not-run/null/zero and valid units retain their semantics.
- Keep the published filtered baseline separate from full raw histories. Never collapse concurrent jobs into logical builds without an explicit collision-tested mapping.
- Native replay must explicitly declare job isolation/order/cost/feedback/test-label assumptions. Reject unsupported protected temporal evaluation rather than inventing clocks or full feedback.
- Run the actual installed wheel outside checkout through real source conversion, native execution and checkpoint continuation twice. Document original/source-order and temporal eligibility or reproducible blockers.
Dependencies
Existing implementations remain the starting point: registry/profiles/materialization, RTPTorrent/LRTS readers, 4.7 LRTS ETL, controlled/JUnit producer, native policies/API/checkpoints and #251–#260 foundations. Inspect the actual code before proposing APIs. This issue covers only the remaining work above; already delivered slices are not reopened.
Shared acceptance gates
- Reuse the existing runner, RunSpec, cache, results, immutable artifacts and checkpoints; keep the domain independent of CLI/MCP/network transports. No unrelated refactor or replacement framework.
- Version new contracts; preserve legacy semantics, explicit units/missingness, deterministic identities/serialization and strict future-version rejection.
- Include meaningful unit/golden/compatibility/property or metamorphic tests as applicable, interruption/concurrency/idempotence/recovery checks, and a real installed-package workflow outside checkout without mocking the principal component. Normal tests stay network-free with owned tiny fixtures.
- Human documentation and Mermaid where appropriate must describe actual usable behavior, terms, provenance and evidence limits. No foreign large data/code is vendored.
- Before push: full merge-base review, OpenCodeReview 1.12.13 local delegation/rules plus manual coverage of all exclusions, all relevant tests/lint/format/types/Pylint/strict-docs/package checks and git diff --check. Fix confirmed critical/high/medium findings, review the exact pushed head and resolve valid PR threads before merge.
- Publish actual source/result evidence in the closing PR. Seeds do not establish independence; negative/inconclusive results remain valid.
Activity
jacksonpradolima commentedon Oct 16, 2026
Implementation is ready in #315, targeting
release/4.8after #301/#314. Local review covers every changed file; full suite: 2,533 passed, 95.30% coverage. Actual real-source installed workflow preserves 46,995 observations / 1,662 jobs, projects 18 executions / eight modeled rounds, and resumes Random/UCB with 16 result rows. PR contains source/receipt/producer digests, commands, compatibility and remaining temporal limitations. Waiting for exact-head review and checks before integration; parent #261 stays open.jacksonpradolima commentedon Oct 16, 2026
Completed through #315, squash-integrated into
release/4.8asd475f49c1ad307f54a7d096b7cda6cd115b46997. The integrated tree exactly matches reviewed head36310d8b606d866638844a2eab49aefcacc9d29f.Delivered a versioned complete real-source analytical/native Parquet workflow and existing native replay/checkpoint bridge. Actual installed outside-checkout runs preserve 46,995 observations / 1,662 jobs and 18 executions / eight modeled rounds; Random/UCB resume with 16 result rows. Source/recipe/producer/writer/count/file provenance and migration/error/recovery behavior are documented, with no foreign data/code vendored.
Validation: complete local suite 2,542 passed, 1 Linux-only skip, 95.31% coverage; actual Linux application 76 passed; new real-process regression 10 passed, including a fresh installed base wheel outside checkout. Final test/example cleanup passed 55 tests with the actual RTPTorrent source cache. Full current-head CI passed, fresh review reported no findings/unresolved findings, every thread is resolved, and actual Sonar/open Code Scanning issues are zero. Local OpenCodeReview delegation/rules plus manual review covered 35/35 cumulative changed files. PR includes all commands and detailed evidence.
This closes the bounded ETL/replay child. Parent #261 remains open: the declared source-job serial simulation has
blocked_missing_temporal_evidenceadmission andinsufficient_evidencegeneralization. Publishing 4.8 will follow a separately reviewed integration PR; the current published wheel remains 4.7. Next child is #303, whose public-source preparation is recorded there.