Add a reproducible controlled non-Java real-fault source #305

Activity

1. jacksonpradolima

   added this to the M5 — Generalizable TCPCI benchmark milestone on Oct 16, 2026
2. jacksonpradolima

   added a parent issue on Oct 16, 2026

   • Build a diversified, versioned benchmark corpus for adaptive CI test prioritization #261
3. jacksonpradolima

   marked this as blocking on Oct 16, 2026

   • Integrate evidence-backed source admission and production-reality coverage #309
4. jacksonpradolima commented on Oct 17, 2026

   

   jacksonpradolima

   on Oct 17, 2026

   OwnerAuthor

   More actions

   Pre-execution protocol for this bounded source qualification (no PySnooper subject tests have been executed yet).

   This evaluates one curated public fix in two predeclared environments. It does not establish benchmark admission, independent faults, historical CI coverage or algorithmic benefit.

   • Subject: public cool-RR/PySnooper; buggy e21a31162f4c54be693d8ca8260e42393b39abd3, its direct fixed child 56f22f8ffe1c6b2be4d2cf3ad1987fdb66113da2 (upstream issue 124). BugsInPy 11c5f1eea954a42132cfd06bf257766a7963e0fd is a curation reference only. No BugsInPy framework code, patches or fixtures are copied.
   • Production: all five original pysnooper/*.py files and LICENSE per revision, with unchanged bytes. Four files have explicit MIT headers; variables.py has no separate header and retains the project LICENSE.
   • Common fixed-tree fixture: unchanged tests/__init__.py, tests/utils.py, tests/test_chinese.py and LICENSE. Select exactly tests/test_chinese.py::test_chinese in both roles. This is an explicitly partial upstream suite; no source/header/test edits, monkeypatches or principal-component mocks.
   • Official CPython 3.8.1, native linux/arm64, image sha256:ffae7d875975e38d38378ce65d10ab62c2156eb198e2e798664ec6c793015c03. Image-provided distributions: pip20.0.2, setuptools45.1.0, wheel0.34.2. Metadata-only probes already confirmed the interpreter, architecture and both locales; no subject was mounted in those probes.
   • Freeze both LC_ALL/LANG contexts before outcomes: C / ANSI_X3.4-1968 / filesystem ascii, and C.UTF-8 / UTF-8 / filesystem utf-8. TZ=UTC, PYTHONUTF8=0, PYTHONCOERCECLOCALE=0, PYTHONIOENCODING=utf-8, PYTHONHASHSEED=0, PYTHONDONTWRITEBYTECODE=1, plugin autoload disabled. Two contexts share one dependence group; they are not independent faults or repeated independent observations.
   • Intervention: replace the original unpinned bootstrap with the declared compatible pytest8.3.5 and eight exact pure wheels below; image-provided setuptools is not replaced. Offline pip --isolated --no-index --no-deps --no-compile, no project setup.py or source builds. Preserve every wheel byte, including unused modules and the explicitly pinned PythonToolbox .data/scripts member; pip may install but does not run that script.
   • Guards per role: no container network; read-only filesystem; non-root; all capabilities dropped; no-new-privileges;2CPUs;64PIDs;1GiB memory;128MiB tmpfs;60-second parent deadline;1MiB combined stdout/stderr. No automatic image pull. Cache, raw packets and outputs physically outside Git.
   • Observation: real pytest setup/call/teardown outcomes and qualified exceptions; integer monotonic ns distinct from parent-process ns and from scheduling costs; actual UTC microsecond collection clocks. Bounded raw bytes retained. Zero stays observed, missing clocks stay unavailable, skip/error/cancellation/incomplete remain explicit.
   • Generic classification is unchanged: only assertion_failed to passed is expected. Non-assertion test_error to passed is inconclusive even if compatible with the curated fix. Never relabel an exception to improve counts. No observed result has been used to alter the test universe or locale selection.
   • Durability: write frozen plan and per-role ownership before execution, role packets after cleanup, completion last only for both valid protocols. Same-trial replay performs no new observation; started/uncommitted roles recover as interrupted, requiring a new trial ID for fresh collection. Retain all failed/incomplete attempts.
   • Terms: source/fixture MIT reviewed independently. Wheels retain complete original bytes and separately pinned component notices. PythonToolbox is not blanket MIT: Envelopes MIT, SortedContainers Apache2, decorator BSD2, two unused GUI modules wxPython/wxWindows3.1 (full pinned upstream licence/LGPL/GPL/preamble retained). Its unused headerless forked_mapping_tests has source similarity to CPython3.3.0; attribution is explicitly inferred, not established origin, and the full pinned CPython LICENSE is retained conservatively. This limitation remains visible for later admission review. exceptiongroup retains MIT+PSF notices; packaging retains both Apache2/BSD2 texts; typing_extensions retains its full historical Python license text.
   • Historical runtime security is outside SDK dependency-audit claims. No foreign source/wheels/outcome data are bundled in COLEMAN; only authored harness and integrity/terms recipes will ship.

   Frozen metadata identities:

   • pysnooper-1-ascii-v1: sha256:8ba330e13824be3e1e361f34caf2e1dad73943f1c81375fc3c3fd580e9efdd65 (63226 resource bytes including final LF).
   • pysnooper-1-utf8-v1: sha256:cad17d9e1594d6a499a627cad09911dc56cd336ee15d8d0af8d71df082b181df (63222 resource bytes including final LF).
   • Authored harness bytes: sha256:e2462a7769167be81bfedea230cff3c3e8249f5f70cb586d9641ab3a1c78b5c2.
   • Deterministic unchanged-file views (buggy/fixed/fixture): sha256:340eac78c4af531ad320df7950df3cef57a905f95c5d4f36dfdaa4aa46318872, sha256:a574c149db26cc8978f4cf8a792aab0421fa2d633737b182b2eb742911171993, sha256:4164070251fe0294eb66df917b947c6b39612938f27693b41d8850a64933b0fe.

   Frozen whole-wheel identities:

   • exceptiongroup==1.3.1: sha256:a7a39a3bd276781e98394987d3a5701d0c4edffb633bb7a5144577f82c773598, 16740 bytes, exceptiongroup-1.3.1-py3-none-any.whl.
   • iniconfig==2.1.0: sha256:9deba5723312380e77435581c6bf4935c94cbfab9b1ed33ef8d238ea168eb760, 6050 bytes, iniconfig-2.1.0-py3-none-any.whl.
   • packaging==26.2: sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e, 100195 bytes, packaging-26.2-py3-none-any.whl.
   • pluggy==1.5.0: sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669, 20556 bytes, pluggy-1.5.0-py3-none-any.whl.
   • pytest==8.3.5: sha256:c69214aa47deac29fad6c2a4f590b9c4a9fdb16a403176fe154b79c0b4d4d820, 343634 bytes, pytest-8.3.5-py3-none-any.whl.
   • python-toolbox==1.3.2: sha256:0f8a0e461755222a9815f0097230341d0aefb16e42f8e56a3fdaae3e2260abdc, 525474 bytes, python_toolbox-1.3.2-py2.py3-none-any.whl.
   • tomli==2.5.0: sha256:32a7b79ac57a2e83670ce329ccf675798bc5a2094783a63676866b70503f2e2b, 14765 bytes, tomli-2.5.0-py3-none-any.whl.
   • typing-extensions==4.13.2: sha256:a439e7c04b49fec3e5d3e2beaa21755cadbbdc391694e28ccdd36ca4a1408f8c, 45806 bytes, typing_extensions-4.13.2-py3-none-any.whl.

   Qualification will run the installed package outside checkout, preserve real packets for both locales, revalidate/replay offline, and exercise the existing native CI/checkpoint flow separately. Controlled case durations will not be invented as native CI execution costs. Complete public recipe metadata will accompany the reviewed implementation.
5. jacksonpradolima commented on Oct 17, 2026

   

   jacksonpradolima

   on Oct 17, 2026

   OwnerAuthor

   More actions

   Observer-bootstrap amendment before qualification of the hardened producer in PR #322.

   The subject, buggy/fixed revisions, common unmodified fixture, selected test, eight whole wheels/component notices, image, locale contexts, timezone/flags, resource guards and strict result classification remain exactly as frozen in the original protocol. No source/test/locale selection or outcome definition is changed. Original qualification packets and their original producer are retained unchanged; old observations are not migrated or counted as new independent evidence.

   The Sonar review pointed at a dependency-directory literal. That literal described a path inside a container-private tmpfs, not a host allocation. The bootstrap is nevertheless hardened to allocate an actual private temporary directory atomically (mode0700), use it for offline wheel installation/imports, and clean it after observation. Its plan now declares the explicit private-runtime-tempdir-v1 storage policy instead of a predictable directory path. An owned real-harness regression verifies the directory exists with mode0700 and is removed afterward. It failed against the original bootstrap and now passes. No foreign subject has been executed with this updated harness yet.

   Other review changes isolate parser/notice preparation helpers without changing their decisions, replace repeated path constants, and precompute exception-test fixtures so only the intended validator is inside each raises block. The initial-head Codex review completed with both findings flags false; a fresh review and all gates will be required for the updated head.

   Freeze the following updated observer/recipe pins before its first foreign-source qualification:

   • pysnooper-1-ascii-v1: sha256:845689cd8cb88437004a8829f35fee164a85375ad532b06a4687d185cda36e8d.
   • pysnooper-1-utf8-v1: sha256:8362d695811ff9324df8d2275e758986ec8e023a42fe86c3e44ffff5c34d1eab.
   • COLEMAN-authored harness: sha256:dde726da7ed5c0a615b12d856f8032931e3e72fa52c24e92f4f87b741e01a1b0.

   Both recipes differ from their original public metadata only in harness_sha256. All source-view, fixture, test, wheel, component, image and locale declarations are byte/field-identical. The baseline source-selection/dependence/negative-result limits remain unchanged. Updated qualification will rerun both predeclared contexts, installed-package validation, actual interruption recovery and the separate existing native CI/checkpoint workflow. It establishes observer correctness, not independent statistical replication or corpus admission.
6. jacksonpradolima

   added a commit that references this issue on Oct 17, 2026

   feat(corpus): add reproducible controlled Python fault capture (#322)

   ...

   580262d
7. jacksonpradolima

   closed this as completedin #322on Oct 17, 2026