Integrate evidence-backed source admission and production-reality coverage #309

Activity

1. jacksonpradolima

   added this to the M5 — Generalizable TCPCI benchmark milestone on Oct 16, 2026
2. jacksonpradolima

   added a parent issue on Oct 16, 2026

   • Build a diversified, versioned benchmark corpus for adaptive CI test prioritization #261
3. jacksonpradolima

   marked this as blocked by 7 issues on Oct 16, 2026

   • Deliver real RTPTorrent Parquet ETL and native replay #302
   • Expand LRTS source slices and resolve stage and temporal admission evidence #303
   • Build a bounded controlled Java real-fault matrix #304
   • Add a reproducible controlled non-Java real-fault source #305
   • Add a permitted cross-language seeded-fault benchmark slice #306
   • Expand HCS topology coverage and independent variant accounting #307
   • Implement the deterministic interaction-covering CI stress laboratory #308
4. jacksonpradolima

   marked this as blocking 4 issues on Oct 16, 2026

   • Validate corpus metric references and protected repeated-measures protocols #310
   • Run a comparable baseline matrix with explicit external-method eligibility #311
   • Assess public GitHub Actions or GitLab prospective collection feasibility #312
   • Freeze and execute the confirmatory corpus and publish the cross-corpus report #313
5. jacksonpradolima commented on Oct 18, 2026

   

   jacksonpradolima

   on Oct 18, 2026

   OwnerAuthor

   More actions

   Pre-implementation protocol for #309. Dependencies #302–#308 are delivered; #308 source #328 and documentation #329 are integrated, the actual signed 4.14 wheel and deployed documentation were qualified. This freezes the rules before computing new admission results; it is a plan, not a result.

   Purpose: produce one evidence-backed source-admission and production-reality view using the existing CorpusRegistry/DataCard/manifests, native profiles, source-specific completion receipts, SourceCache, ArtifactWriter and independent temporal contracts. Do not replace the runner or legacy phase-one readiness. The default registry/card artifacts and all historic result semantics remain readable unchanged.

   1. Freeze source bindings before computing an admission outcome. Every actual input needs source/receipt/schema/producer/writer identity and actual cache/output verification through its existing source-specific validator. Mere registration, child closure, metadata-only labels and claimed self-digests cannot qualify a source. Retain explicitly unavailable/rejected/planned sources.
   2. Reuse the existing CorpusRegistry and DataCard types for the integrated inventory. Preserve original registry cards/IDs and record any new derived card version/identity explicitly. Do not silently relabel a legacy planned native adapter as implemented. Keep source-specific component terms, pins and redistribution rules; a report's own license cannot clear its inputs.
   3. Separate purposes: historical/protected temporal evaluation, controlled functional reproduction, seeded/constructed evidence and deterministic stress mechanism checks. An admitted functional or synthetic slice cannot fill an observed historical CI cell. Passing source integrity alone is insufficient for any scientific admission; articulate the exact additional source-specific checks. Negative/both-pass/both-fail/inconclusive observations remain eligible evidence, never dropped because they do not support a desired result.
   4. Historical clocks must establish event, label and feature availability before decisions, along with expected-universe/identity/group rules and protected protocol. RTPTorrent job labels, scheduled LRTS timestamps and HCS pipeline order remain insufficient when actual availability is missing. Do not manufacture missing clocks or turn reported serial durations into parallel latency/work savings.
   5. Keep row/test counts separate from source project/dependence groups, curated fault/context groups and genuinely mapped unique faults. Multiple Log4j stage views share project/build evidence; HCS variants and stress copies share logical tests; locales/repeated trials/seeds do not add independent projects/faults. Curated fault labels alone do not establish a unique-fault mapping.
   6. Production matrix must include languages/build systems, provider or execution model, triggers/test levels, setup/shared fixtures/order dependence, serial/parallel/matrix/shard/rerun scope, runner/queue/concurrency/cancellation, retries/quarantine/flakiness/timeouts/infrastructure failures, vendor/monorepo/renames/change-to-test traceability, protection/mandatory suites/overrides, partial/omitted feedback, delayed/corrected outcomes and retention. Each fact explicitly observed/reconstructed/missing/synthetic (or inapplicable with reason), with a source-bound locator. Unknown is not zero, unsupported is not absent, source labels do not certify actual runtime topology.
   7. Reuse explicit units/Measurement/NumericProfile where applicable. Compute descriptions from actual verified rows, and distinguish reported label/ordering assumptions from observed CI continuity. Keep non-finite, negative, zero, null/unavailable and non-applicable semantics explicit. Churn in unverified label order cannot become observed chronological churn.
   8. Deterministic versioned artifacts, bounded metadata/rows/bytes, future-schema rejection, no implicit migration, atomic complete-last publication, idempotent retries and conflict rejection. Reading checks structural/derived invariants; explicit validation/publication rechecks actual external bytes. No CLI/MCP/provider imports in the domain and no network/subject execution during offline admission.
   9. All seven required regimes remain in the matrix. Admit a cell only with passing source-specific evidence for its stated purpose; any failed admission stays uncovered and blocks final parent closure. Represent two materially different source CI models without flattening original project/job/stage/pipeline identities. No predetermined admitted count or efficacy claim.
   10. Qualification: meaningful contract/golden/round-trip/legacy/futureversion/units/missingness/negative/nonfinite/property/metamorphic/tampering/leakage/dependence tests; actual interrupted publication and retry/concurrency; installed wheel outside checkout with transport import guard; at least one real existing runner/checkpoint flow and actual available public-source artifacts verified offline. Normal tests network-free/owned tiny fixtures. Full frozen suite, lint/format/types/Pylint/strict docs/build/install/diff, all-file local Alibaba1.12.13 rules/manual review, exact-head PR reviews/checks before normal merge.

   The issue and all comments were refreshed (no comments at this point), current main is clean at 5c934d55c5957cef4f9557002eae3bee5a72e3d6, and ancestor/repository AGENTS checks found no applicable files. Existing contracts and actual prior source artifacts were inspected; begin with their existing offline validators. Do not weaken admission requirements to close #261. Remaining source/evidence blockers belong within the existing children/parent, not duplicate issues.
6. jacksonpradolima commented on Oct 18, 2026

   

   jacksonpradolima

   on Oct 18, 2026

   OwnerAuthor

   More actions

   Source-verification clarification before the complete corpus qualification:

   The generic controlled converter validates pinned source/report bytes and observation accounting, but it does not reparse producer-specific raw packets. That is insufficient by itself for controlled-functional source admission. The existing capture producers also bind their entire installed SDK and package version; unchanged individual legacy modules do not authorize relabeling a capture to a newer release.

   The development implementation therefore requires an explicit typed raw-producer verifier binding for a positive controlled-functional purpose. The domain records a versioned proof and re-invokes that binding during source validation/publication; reading proof metadata alone grants no verification. The separate application adapter checks a caller-pinned original wheel, every installed SDK resource/version and invokes the original package's existing Python/C/Java-matrix raw-wire/source/context validators offline. Its interpreter/plan/receipt/output paths are local bindings, excluded from portable identity. The portable proof retains original wheel/version/producer, raw packet digests, receipt/plan identity, verifier code identity, Python and dependency versions. No network, Docker, source test execution, automatic installation or old-artifact rewrite occurs; existing validators may republish identical owned cache metadata. The original wheels remain available from reviewed public releases.

   The domain imports no CLI/MCP/capture transport. Metadata-only controlled inputs remain blocked with original_raw_producer_verification_missing. Actual original-wire correspondence still does not prove authentic physical execution, legal permission, independent faults or policy efficacy. Human source/terms/curated-fault review remains separately explicit.

   An initial development check revalidated the previously collected PySnooper ASCII evidence with its original published 4.13 wheel and full original producer identity. Both raw role packets and normalized observations agree; the test-error/pass pair remains inconclusive, qualifying only for the reviewed functional purpose. It does not become historical CI or an assertion-failure detection. This is not the complete corpus qualification or a released 4.15 API. Full frozen tests, actual complete-source evidence and local/PR review remain required before delivery.
7. jacksonpradolima

   closed this as completedin #330on Oct 18, 2026