Skip to content

F1 + F2 follow-ups: fail-closed demo mode, computed cost fallback, first tests, CI#2

Merged
sgharlow merged 3 commits into
mainfrom
followups/f1-f2-ci
Jul 18, 2026
Merged

F1 + F2 follow-ups: fail-closed demo mode, computed cost fallback, first tests, CI#2
sgharlow merged 3 commits into
mainfrom
followups/f1-f2-ci

Conversation

@sgharlow

Copy link
Copy Markdown
Owner

Follow-ups from the PR #1 review, resolved for maximum credibility rather than minimum patch:

F1 — undisclosed demo output → structurally impossible

  • DEMO_MODE !== 'true' → the PR_ANALYSIS job fails with an explicit reason and posts nothing. A default deployment can no longer present fabricated findings — safety by structure, not by label.
  • With demo mode on: the summary comment opens with a DEMO OUTPUT banner, the heading says "(demo sample)", and every inline comment self-identifies.
  • Proven by test: an exploding-proxy GitHub client (any property access throws) passes untouched through the disabled path.

F2 — the fail-closed pricing claim is now computed, not asserted

  • Fallback = max over the pricing table (was hardcoded gpt-4 while gpt-4-32k costs 2×). Test asserts the fallback ≥ every listed model, so the guarantee can't silently rot when prices are edited.

First tests + CI (closes deferred Sprint-5.1)

  • node:test over compiled dist — zero new dependencies; 3/3 green locally.
  • CI: build + tests on Node 20/22, Docker build validation. This repo went from never-compiled to compile-gated in one day; this PR makes regression impossible to miss.

README

sgharlow added 3 commits July 18, 2026 12:11
- F1: demo processor is fail-closed — refuses without DEMO_MODE=true (no
  GitHub call possible before the gate, proven by test with an exploding
  client proxy); when enabled, the summary comment and every inline comment
  are prominently labeled as canned demo output
- F2: unknown-model pricing fallback is now COMPUTED as the most expensive
  listed model (was gpt-4 while claiming max — gpt-4-32k costs 2x); the
  fail-closed property is asserted by test against every listed model
- First tests in this repo (node:test over dist, zero new deps) + CI:
  build+test on Node 20/22, Docker build validation (closes the deferred
  Sprint-5.1 item)
- README banner updated: repaired state, benchmark case-study pointer,
  honest claim level (built, not live-proven)
…ev then ran tsc (a devDependency); it could never build. Build stage compiles, runtime ships prod deps + dist
@sgharlow
sgharlow merged commit 3abf4c8 into main Jul 18, 2026
3 checks passed
@sgharlow
sgharlow deleted the followups/f1-f2-ci branch July 18, 2026 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant