Repository navigation
feat(corpus): release reproducible controlled CI evidence for 4.6 - #296
Conversation
…294) Add versioned offline controlled fault contexts, source-bound observation reports and deterministic immutable artifacts. Preserve expected, unexpected and inconclusive paired outcomes with exact elapsed-counter units and pinned diagnostics. Reuse native source caches, runner, manifests and checkpoints. Refs #261; source admission and the diversified benchmark program remain open.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1283fba534
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review Current release head d3f804d addresses both P2 findings #4295623296/#4295623307 and the related nested/cache/staging Git-boundary cases. Fifteen regressions failed before their fixes; 238 focused checks and the complete 2,258-test suite now pass, with 95.26% combined coverage and actual installed Java collection/recovery. All quality/package gates and repeated local full-range delegation/rules/manual37/37-file review passed. Please review the entire current release diff. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d3f804d2eb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review Current head 80e6681 fixes envelope-size P2 #4296021393 while preserving complete raw packets, source-view limits/golden bytes and archive policies. A valid4MiB large-diagnostics regression now publishes/replays >8MiB role records/>16MiB envelope without rerunning roles; normalized-report combinations preflight before execution. Final validation:2,268 tests,95.28%combined coverage,317focused tests,actual installed95-per-role source flow, allquality/package gates and repeated local-only full-range/manual38/38-file review. Please review the complete current release diff. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 80e668164b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
@codex review Head d1d0a1e fixes discovery-output P2 #4296394857 with two real JVM red/green regressions, preserved bounded raw stderr, excluded-class noninitialization and an explicit harness repin. Final validation: 2,270 tests, 95.28% combined coverage, 71 focused real/installed checks, all quality/package gates and repeated local-only full-range/workspace/rules/manual39/39-file review. Please review the entire current release diff. |
|
Codex Review: Didn't find any major issues. 🎉 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Reproducible controlled CI evidence
Refs #261. Groups reviewed #294 and #295 for COLEMAN 4.6. CI users can trace controlled buggy/fixed observations to pinned source, a common fixture, dependencies, runtime and actual producer bytes; retain raw diagnostics and negative outcomes; and recover collection without inventing completion.
The installed recipe reproduces 190 observations / 95 tests per revision: 92 both-pass pairs, two both-fail pairs and one failure-to-pass pair. This is functional reproduction, not observed CI history, an independent-fault count or a policy-performance comparison. #261 remains open.
Contract, architecture and compatibility
Usage
With the documented prepared arm64 Temurin image and installed wheel:
The second call reopens the same observations without reexecuting completed roles.
docs/junit-source-capture.mddocuments complete preparation/terms/recovery;docs/controlled-faults.mdprovides the SDK and owned native TCPCI example with independent metrics/checkpoint continuation.Validation and local review
Final head
d1d0a1e0fcf1373d2b2eafa28981d8e98b422d92, treeeab11922a01a81757cd0e302abda54b189937b0a: 2,270 tests passed, 13 existing warnings, 95.28% combined line/branch coverage, including actual installed-wheel source execution/recovery. Final focused verification: 71 passed, including two actual JVM discovery regressions and installed 95-per-role execution. Both discovery cases failed before the fix. Other confirmed storage, counter and envelope findings have red/green regressions, including large-packet publication/replay without reexecution.PATH=/tmp/coleman-4-syft-tools:$PATH \ COLEMAN_JUNIT_REAL_CACHE=/tmp/coleman-261-junit-recipe-source-cache \ uvx --from uv==0.13.1 uv run --no-project --python .venv/bin/python \ -m pytest -n4 --cov --cov-branch \ --junitxml=/tmp/coleman-296-discovery-tests.xml -o junit_family=legacy UV_NO_SYNC=true make lint format-check typecheck interrogate cost-structural docs build .venv/bin/pylint -E coleman git diff main...HEAD --checkAll listed checks passed. Ruff removed only redundant literal parentheses after the frozen full suite: Python AST equality was verified and the focused real JVM/installed suite repeated. Release-specific Syft inventory, note selection, lock/security and generated-reference checks also passed.
OpenCodeReview evidence: used only
@alibaba-group/open-code-review@1.12.13 delegate previewanddelegate rule, never remote review. Repeated dirty-workspace and full--from main --to HEADpreviews; consulted all applicable Python/Java/JSON/default rules; manually reviewed 39/39 changed files, including all 21 exclusions, tests, Markdown, generated docs and JSON values. Previous reviewed bytes were hash-verified; the entire release delta and interactions were reviewed. Local evidence:/tmp/coleman-296-discovery-{range,final-workspace,rules,manual-review}.json; full/quality/focused/red/green logs use the same prefix.Release handling and limits
The package/lock stay at 4.5.0 in this PR. A compatible
feat:squash to main lets the existing workflow select 4.6.0, synchronize package/lock before tagging, build/sign distributions and publish reviewed en-USdocs/releases/4.6.0.md. No manual bump or dispatch. Verify published digests, tag/source/wheel/lock/SBOM, signatures and actual installed workflows afterward.One direct upstream Java pair, shared JVM and lexical order; no established order independence or full Defects4J reconstruction. Runtime guards support trusted-source reproduction. All outputs retain
not_evaluated/insufficient_evidence. Independent admission, non-Java sources, seven diversity regimes, comparable baselines, protected chronology and cross-corpus confirmation/report remain sequential #261 work.