Repository navigation
feat: add bounded controlled Java fault matrix - #320
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
@codex review Review exact head |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
Codex Review: Didn't find any major issues. More of your lovely PRs please. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@codex review Please review exact head |
|
Codex Review: Didn't find any major issues. Bravo. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@codex review Please review the complete PR at exact head |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 87167b778a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review Please review the complete PR at exact head |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 30f180d97a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review Please review the complete PR at exact head |
|
|
Codex Review: Didn't find any major issues. Already looking forward to the next diff. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
2 participants
COLEMAN could reproduce one pinned Java fault pair. This adds a bounded, predeclared CLI/Codec source matrix with typed plans/indices, offline integrity validation and durable recovery, reusing the existing JUnit producer and generic controlled-observation domain.
Closes #304. Part of #261; source admission, diversity coverage, baselines and protected confirmation remain open in #309–#313.
Contract and architecture
ControlledDataset, immutable artifacts and existing native CI/checkpoints. No existing producer/domain implementation or original CLI-13 recipe changes; no new dependency or learning framework.plan.jsonbefore source execution. Publish a content-addressed index and writecomplete.jsonlast only when every role was collected. Incomplete evidence remains indexed without completion; unattempted cells remain in the frozen plan.Sources and scientific limits
Selection was recorded before new source outcomes.
Actual source qualification collected 714 role observations / 357 same-test pairs. CLI-51 raises
UnrecognizedOptionExceptionin the buggy role and passes in the fixed role; the existing strict classifier retains this as inconclusive. Both CLI fixtures retain two assertions failing in both revisions. Three Codec triggering methods belong to one curated fix; the CLI fixes share project history. Labels, tests and repeated trials do not establish independent projects/faults.Codec keeps all 25 production Java files per role, 15 selected fixed fixture Java files, LICENSE and NOTICE. Before outcomes, excluded DoubleMetaphoneTest (separately noticed aspell data/non-UTF8 bytes), SoundexTest (non-UTF8 bytes), and their AllTests aggregator. RefinedSoundexTest remains included. Source bytes are not transcoded and no post-outcome test filtering occurs. These are direct-upstream parent/fix reproductions, not full upstream build/Defects4J reconstruction.
The conscious annotation correction and old/new descriptor fingerprints record that subsequent source inspection identified unseeded
java.util.Randominputs in Base64Test, HexTest and DigestUtilsTest. All selected tests remain; the uncaptured generated inputs/seeds are explicitly disclosed in recipes and documentation. A provenance-only annotation correction preserves every source/dependency/runtime/harness pin and test ID. Replaying stored observations is distinct from exactly reconstructing fresh generated test inputs.The pinned Temurin 11.0.32.1/linux-arm64 image, explicit America/Los_Angeles timezone, common fixture and raw JAR/terms pins remain visible. Per-role process time differs from case time, runner work, memory and CI latency. The fixture shares a JVM/lexical order; order independence is unverified. All outputs remain constructed,
not_evaluatedandinsufficient_evidence.Usage and compatibility
The complete guide includes external wheel installation, explicit image preparation, source acquisition, CLI/Python calls, source terms, Mermaid architecture, recovery and installed verification. With the reviewed wheel installed and pinned image prepared:
Source/cache/output storage stays outside Git. Fetch is explicit and uses the existing bounded public HTTPS transport; subject execution has no network. The producer never pulls an image. Large foreign sources/dependencies/captures are not vendored.
Existing single-pair recipes, controlled artifacts, native results, RunSpec identities and learning checkpoints require no migration. New matrix schema readers reject missing/older/future versions; unknown producers reject with an explicit compatibility requirement. Changed implementation/package version selects a new generation; old bytes are not silently rewritten. Offline validation needs the matching reviewed producer plus actual controlled datasets/cache, not just an index.
How this was tested
Final exact-head full suite: 2,680 passed, 1 Linux-only skip, 13 existing warnings; 95.27% branch-aware coverage in 835.27 seconds. HEAD
8938121e70c677ff725a144ea27ccb32b6a8a661and all source/test bytes remained unchanged throughout.Commands:
The full run enables the prepared real Java, LRTS, RTPTorrent ZIP64/ETL and LRTS stage caches; Syft 1.42.3 is available for release-package checks. Normal owned tests remain network-free. Additional evidence covers:
llms.txt, source metadata/pin audits and the actual release-workflow selector for 4.10.0 notes. No manual bump; reviewed EN-US release description is included.Test configuration: Python 3.14.6/macOS arm64 host, pinned native Linux arm64 Temurin 11.0.32.1 Docker image; pytest 9.0.3, uv 0.13.1. Hardware-specific performance claims are not made.
Local review
OpenCodeReview 1.12.13 local delegation only:
npx --yes @alibaba-group/open-code-review@1.12.13 delegate preview \ --format json --from main --to HEAD \ -b 'Implement public issue #304: bounded controlled Java real-fault matrix with pinned recipes, explicit negative evidence, raw provenance and offline recovery'Also ran workspace preview and
delegate rulefor every changed path. Six supported files plus twelve exclusions were manually reviewed, including every test, Markdown file, golden and generated artifact: 18/18 files, 100% file review coverage. JSON values received separate typed/source-pin review beyond the tool's key-only rule. No code was sent to an external review endpoint. All confirmed blocking findings were corrected; review was repeated after material changes. Evidence:/tmp/coleman-304-complete-local-review.json, local preview/rules/full-diff records, source-disclosure proof and final frozen-source stability proof.PR review follow-up: the new CLI validation mode accepts bounded receipt JSON from stdin, not a filesystem path. Filepath arguments (including parent/symlink aliases) are rejected without opening their files; oversized stdin rejects before parsing. Typed Python validation and all source collection logic remain unchanged. Updated the installed real-package CLI workflow and guide, repeated the entire 18-file local delegation/manual review and all exact-head checks. The repeated-literal and split-assertion findings remain addressed. No scanner rule was suppressed.
PR provenance follow-up: replaced the manually maintained module allowlist with a deterministic traversal of all COLEMAN packaged code/resources, without importing them. This includes registry metadata, ZIP metadata, temporal contracts and future resource formats. Hidden files, generated bytecode and interpreter caches are excluded. Conservative invalidation means even unrelated packaged changes select a new generation; the guide/release explain this compatibility limit. Two additional transitive regressions reproduced the old digest; all60focused tests and the actual installed three-source Java workflow pass. Exact checkout/wheel byte comparison covers all260packagefiles. No existing JUnit/domain bytes changed.