P1: Collect and execute truthful opt-in relevant pytest tests

[tomdps]

Parent

Tracking parent #211; epic #243. This is one vertical delivery slice. Do not close or bypass the parent tracker from this child.

Current dev prerequisite truth

• P0 — Harden Python type checking and configuration semantics #245 is closed after the authoritative mypy/Pyright children.
• P0 — Resolve Python projects, interpreters, environments, and tools canonically #246, fix(python): share import resolution between validation and graph #209, and Hypothetical (validate pre-write) evaluation must be comprehensive — graph-backed checks are silently skipped #197 are accepted and merged.
• P1: Run truthful opt-in Ruff lint and format checks #258 is merged through PR feat: P1: Run truthful opt-in Ruff lint and format checks #283 at 83f9bcef9b6419b092ec252eae299ca8c6936763.
• Implement from a fresh origin/dev worktree. PR base is dev.

Goal

Separate graph test candidates from real pytest collection/execution and run only explicitly enabled relevant tests with bounded, exact after-state evidence. Graph relevance chooses candidates only; fresh pytest collection and execution remain the test authority.

Required behavior

1. Retain python.relevant-tests as graph candidate selection only. Rename candidate codes/messages and metrics away from untested/pass wording. Add separate opt-in, graph-required python.pytest with defaultScopes: []; unrequested is not_applicable, disabled is disabled, and neither state invokes a process or degrades unrelated capabilities.
2. Select exact repo-relative candidate test files only from accepted Hypothetical (validate pre-write) evaluation must be comprehensive — graph-backed checks are silently skipped #197 after-state graph sessions and fix(python): share import resolution between validation and graph #209 TESTED_BY facts. Stale/provider failure is infrastructure failure. Missing candidates, missing/deleted candidates, zero collection, zero execution, and no actual passes are typed non-pass outcomes—never clean success or a whole-repo/test-directory fallback.
3. Reuse P0 — Resolve Python projects, interpreters, environments, and tools canonically #246 canonical per-target Python project/interpreter/config context. Keep config-only requests inside the config's canonical project, remove deleted sole boundaries from after-state discovery, and group nested/mixed targets without cross-project execution.
4. Materialize only the repo-confined visible after-state project/source/config/support/test files required by the selected project. Preserve exact UTF-8 bytes and regular-file mode, including staged/tree reads independent of dirty worktree mode. Refuse unsafe symlinks, special files, undecodable content, source/config deletion contradictions, workspace file/byte caps, and path escapes before pytest execution.
5. Load an Opcore-owned zero-dependency pytest hook source from the temporary runtime. Disable ambient plugin autoload. The hook emits bounded strict JSON-lines collection node IDs and setup/call/teardown outcomes. Collection and execution run in separate fresh temporary workspaces so collection mutations cannot affect execution.
6. Execute exactly the collected parameterized node IDs. Direct argv is allowed only under a fixed byte cap; otherwise write a bounded owned selection manifest and pass no raw node IDs. Bound candidate count, collected-node count/identity bytes, manifest bytes, hook-report bytes, process output, subprocess duration, and total workspace size. No truncation may become success.
7. Reconcile session, collected, executed, passed, failed, skipped, xfailed, xpassed, and error counts strictly. Collection/execution exit codes, malformed/truncated/duplicate/unknown hook events, contradictory counts, timeout, signal, spawn failure, output overflow, invalid config, no tests, all-skipped/no-pass, and cleanup failure remain distinct typed outcomes. Preserve earlier sibling-project receipts and diagnostics.
8. Emit portable PythonValidationCapabilityRun evidence with candidate paths separate from collected IDs, separate collection/execution invocations, redacted execution argv plus digests, selection mode/digest, exact tool/version/config/cwd, after-state fingerprint, process termination/timing, counts, and mandatory cleanup evidence. Host/workspace/temp paths and parameter values must not leak into public receipts.
9. Wire contracts, JSON schema, validators, registry/activation/status, metrics/reporting, ASP assessment mapping, package exports/packlists, docs, and installed public behavior to the same truth. passed requires one or more fresh passing tests, successful collection and execution, exact count reconciliation, and completed cleanup.
10. Validation must not install/sync/download pytest, create project environments, mutate source/config/lock/environment/cache files, retain temporary resources, run arbitrary unselected tests, or claim sandbox/security/readiness/parity.

Acceptance matrix

• Activation and policy: unrequested, disabled, enabled, missing pytest, graph unavailable/stale, and sibling-project mixed outcomes.
• Discovery and grouping: explicit source/config targets; pytest.ini, pyproject.toml, tox.ini, setup.cfg; nested projects; config-only requests; added/modified/deleted/renamed source, test, config, and conftest inputs.
• Selection: no candidates, missing/deleted candidate, one/many candidates, parameterized node IDs, direct argv, bounded manifest fallback, candidate/node/workspace/report/output limits.
• Results: pass, assertion failure, setup/import/collection errors, skip, xfail, xpass, all-skipped/no-pass, exits 2–5, timeout, signal/spawn, malformed/truncated/duplicate/mismatch protocol, and cleanup failure.
• Exact state: collection mutations cannot affect execution; executable file modes and staged/tree modes are preserved; source/config/lock/environment/cache state is unchanged after success and every failure.
• Integration: status/help/registry, metrics, ASP/provider mapping, contract and schema round trips, installed package behavior, and package contents all agree.
• Real authority: a pre-provisioned pinned pytest environment proves clean, failed, collection failure, timeout, large manifest selection, ambient-plugin isolation, cleanup, source immutability, and exact packed-install execution.

Verification

Run focused behavior first, then repository and installed proof:

• npm run build
• node --test tests/validation-python.test.mjs
• node --test tests/contracts.test.mjs tests/schema-contracts.test.mjs tests/asp-provider.test.mjs tests/opcore-metrics.test.mjs tests/opcore-facade.test.mjs tests/validation-runner.test.mjs
• npm run lint
• pinned real pytest authority proof through scripts/check-python-pytest-authority.mjs
• configured opcore-ci command proof (cmdproof profile ci-equivalent)

Attach exact pytest version, normalized receipts, command/config/cwd/source evidence, immutable-source and cleanup evidence, packed-install proof, PR/commit, and green checks.

Completion rule

Exactly one Opcore implementation ship run may be active. Completion requires a clean scoped PR merged into dev, closed #259, focused and real-tool/packed evidence, configured opcore-ci, and parent #211 checklist reconciliation. A cluster terminal state, opened PR, narrow test, or generated receipt is not completion. Do not claim Python readiness/parity, ASP authority, old-tool replacement, security/SAST, all-stack support, automatic fixes, or a blended score.

Stalled-run recovery evidence (2026-07-25)

• prime-bastion-20 was stopped after 39 implementation iterations and 32 validator rounds because concurrent validator/build commands repeatedly deleted live dist/ artifacts and the run expanded into unrelated CI/build/package rewrites. It did not produce an accepted PR and is not completion evidence.
• Salvage references only: pushed commits 9f698c5 and 0234e52 on origin/zeroshot/prime-bastion-20; preserved uncommitted state is stash@{0} in /Users/tom/.zeroshot/worktrees/prime-bastion-20. Do not cherry-pick either commit wholesale. Re-derive the issue-scoped implementation from fresh origin/dev and use the branch only as optional reference.
• Forbidden recovery scope: no changes to CI workflows, scripts/ci/run-local-ci-equivalent.sh, root/package build orchestration, TypeScript build cleaners, release-receipt generators, checked-in native artifacts, package build commands, or unrelated mypy/Ruff/runtime behavior solely to make concurrent Zeroshot validators pass.
• Run focused proofs first. Run exactly one clean opcore-ci command proof after focused verification; do not launch duplicate/concurrent proof runs in the same worktree.

[tomdps]

Completed by PR #290, merged into dev. Evidence: opt-in python.pytest authority with graph-only candidate selection, canonical project grouping, isolated bounded workspaces, collection/execution mismatch detection, portable receipts, packed-install proof, real pytest authority matrix, focused suites, and opcore-ci result sha256:a19941573a97124b6ff915588b1c713d727fcf74fa433b34d9bb3c74ce9ab7ac (exit 0, no workspace drift). No Python readiness/parity claim is made; release wording remains gated by #251.