Skip to content

fix(ci): align product smoke with runtime API - #611

Open
Lutar, Stephen P. (stephenlutar2-hash) wants to merge 6 commits into
mainfrom
agent/fix-runtime-product-smoke
Open

fix(ci): align product smoke with runtime API#611
Lutar, Stephen P. (stephenlutar2-hash) wants to merge 6 commits into
mainfrom
agent/fix-runtime-product-smoke

Conversation

@stephenlutar2-hash

@stephenlutar2-hash Lutar, Stephen P. (stephenlutar2-hash) commented Aug 13, 2026

Copy link
Copy Markdown
Member

Summary

  • Replace the retired database/session /api/* product smoke with the current Alloy Runtime API contract on the canonical loopback port.
  • Verify build identity, dependency readiness, missing-key rejection, deliberately invalid-key rejection, and an authenticated tenant-scoped read.
  • Wire a per-run loopback-only API key and exact commit SHA into the hosted runtime audit.
  • Promote smoke-product-mode from advisory P1 to blocking P0 so a failed runtime/auth contract fails the harness job.
  • Correct the canonical audit guide's P0/P1, exit-code, lint, trigger, and build/boot descriptions while distinguishing hosted evidence from currently required protection contexts.
  • Escape existing backslashes before Markdown table delimiters and normalize line breaks in step-summary cells.

Scope

Exactly four files:

  • .github/workflows/audit-full.yml
  • docs/ops/audit-harness.md
  • scripts/audit-full.js
  • scripts/qa/smoke-product-mode.js

No package manifest, lockfile, product mutation route, external credential, deployed environment, or branch-protection setting changes.

Local evidence

  • pnpm@10.26.1 typecheck: 182/182 tasks successful.
  • pnpm docs:claims-check: 26/26 claims verified.
  • Node syntax checks: pass.
  • Biome check: pass.
  • Actionlint 1.7.12: pass.
  • Strict workflow YAML parse: pass.
  • git diff --check: pass.
  • Project secret scanner over .github/workflows and scripts/qa: CLEAN.
  • Real production-mode loopback runtime with matching SHA/key: all five probes pass; exit 0.
  • Wrong client key: authenticated probe fails; exit 1.
  • Wrong expected SHA: identity/readiness chain fails; exit 1.
  • Independent mock review also verified missing server key, false readiness, and accept-any-nonempty-key regressions exit 1.
  • Hostile summary-cell encoding matrix (backslash, pipe, CRLF): pass.

Truth boundary

This change proves the ephemeral CI runtime contract only. It does not claim a provider deployment or deployed-runtime witness. Runtime Audit is fail-closed for P0 inside its job but is not currently a required branch-protection context. The pre-existing /readyz probe cleanup defect is intentionally reserved for a separate serialized successor.

Screenshots: N/A (no UI changes).

Probe the current Alloy Runtime API on its canonical loopback port.
Verify build identity, readiness, missing and invalid API keys, and tenant reads.
Make the product smoke a blocking P0 gate with a per-run loopback credential.

Workcell: RUNTIME-AUDIT-P1-20260813
Signed-off-by: Lutar, Stephen P. <stephenlutar2@gmail.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@stephenlutar2-hash

Copy link
Copy Markdown
Member Author

Codex (@codex) review exact head 6b478a9. Please focus on fail-closed P0 propagation, current runtime route/body contracts, API-key negative coverage, secret disclosure, and workflow expression wiring.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Classify the product-mode smoke as blocking P0 inside the harness.
Correct the CI exit semantics and distinguish hosted evidence from the
status contexts currently required by live branch protection.

Workcell: RUNTIME-AUDIT-P1-20260813
Signed-off-by: Lutar, Stephen P. <stephenlutar2@gmail.com>
@stephenlutar2-hash

Copy link
Copy Markdown
Member Author

Codex (@codex) review exact replacement head f268afa. The only follow-up is docs/ops/audit-harness.md, correcting the P0 classification, CI exit semantics, and current branch-protection truth. Please review all four files and disregard the superseded head.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Describe P0 as harness- and job-blocking rather than branch-required.
Match the current Biome lint command, workflow triggers, and build/boot steps.

Workcell: RUNTIME-AUDIT-P1-20260813
Signed-off-by: Lutar, Stephen P. <stephenlutar2@gmail.com>
@stephenlutar2-hash

Copy link
Copy Markdown
Member Author

Codex (@codex) review final replacement head 3f2566b. The latest docs-only commit removes the remaining merge-blocking overclaim and matches the actual Biome lint, triggers, and build/boot sequence. Please review the aggregate four-file diff; earlier heads are superseded.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Use the exact workflow and job names and state that CI skips both the
harness install and E2E steps after installing dependencies separately.

Workcell: RUNTIME-AUDIT-P1-20260813
Signed-off-by: Lutar, Stephen P. <stephenlutar2@gmail.com>
@stephenlutar2-hash

Copy link
Copy Markdown
Member Author

Codex (@codex) review final exact head 10282cd. The final docs-only follow-up uses the exact workflow/job names and accurately states the CI harness skips install and E2E. Please review the aggregate four-file diff; all earlier heads are superseded.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Comment thread scripts/qa/smoke-product-mode.js Fixed
Escape existing backslashes before Markdown table delimiters and normalize
line breaks so runtime check details cannot produce incomplete escaping.

Workcell: RUNTIME-AUDIT-P1-20260813
Signed-off-by: Lutar, Stephen P. <stephenlutar2@gmail.com>
@stephenlutar2-hash

Copy link
Copy Markdown
Member Author

Codex (@codex) review exact head 87d99dd. This signed follow-up fixes the concrete GitHub Advanced Security alert by escaping existing backslashes before Markdown table delimiters and normalizing line breaks. Please review the aggregate four-file diff; prior heads are superseded.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Flatten bare carriage returns as well as LF and CRLF before writing runtime
probe details into the GitHub step-summary table.

Workcell: RUNTIME-AUDIT-P1-20260813
Signed-off-by: Lutar, Stephen P. <stephenlutar2@gmail.com>
@stephenlutar2-hash

Copy link
Copy Markdown
Member Author

Codex (@codex) review exact head e6a0c53. This signed follow-up addresses the independently verified bare-carriage-return CommonMark injection case while preserving the prior backslash/pipe fix. Please review the aggregate four-file diff; prior heads are superseded.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants