Skip to content

fix(build): headless-safe db:push — kill the schema-migration false-green (name-less apps reach green) - #204

Merged
agjs merged 12 commits into
mainfrom
fix/near-green-completion-class
Jul 27, 2026
Merged

fix(build): headless-safe db:push — kill the schema-migration false-green (name-less apps reach green)#204
agjs merged 12 commits into
mainfrom
fix/near-green-completion-class

Conversation

@agjs

@agjs agjs commented Jul 27, 2026

Copy link
Copy Markdown
Owner

The wall this removes

Name-less apps (e.g. bookmarks: title+url, no name) NEVER reached green. Root cause, proven on a live build:

  • The scaffold ships each entity table with a stub name column. A name-less plan makes the model REMOVE name and ADD the real columns.
  • drizzle-kit push can't tell rename from drop+add → hits its interactive column resolver → in the headless build (no TTY) it throws Interactive prompts require a TTY terminal. --force only auto-approves data-LOSS, not this prompt.
  • bun run db:push exits 0 on that crash (async rejection swallowed) → the gate false-greens on a DB that never migrated → every runtime query 500s (column "title" does not exist) → create fails → acceptance parks. Apps that keep a name field slipped through — which is why green was app-shape dependent.

The fix

db-push.ts dbPushForce: run db:push -- --force; detect the crash by its output signature (not the exit code); on that signature only, drop the disposable build entity table over the same DATABASE_URL (the app's own postgres client) and retry a clean CREATE. A swallowed/persistent crash is normalized to a non-zero code so a failed migration can never pass as success; the drop is failure-safe. Wired at generate.ts (throws on failure) and the gate command stage (short-circuits to an actionable db-push-failed error, never running the gate on a stale schema). The redundant, result-ignoring per-feature push in build.ts was removed.

Proof

  • Full bookmarks build → final-acceptance GREEN (browser CRUD, live DB migration to title/url, full validate + prod build + size checks).
  • Integration test migrating the real DB + unit tests across db-push / gate-stages / generate / build (every residual-failure path).
  • Full suite 3114/0. 4-model panel PASS (3 hardening rounds).

Also included

  • Close-on-success form idiom + truthful acceptance steer (panel-passed).
  • Phantom parent-FK fix ('belongs to a User'), completion-class near-green fix (both live-observed).
  • TSFORGE_BUILD_TEMPERATURE lever (experimental, env-gated default-off; temperature was ruled out — safe to revert later).

Gate-preserving throughout.

agjs added 12 commits July 25, 2026 22:00
…ow-UI state

4-model panel (deepseek+glm+grok+codex), cross-examined over 2 rounds, unanimously root-caused
the near-green 'circling'/park: the stuck 1-error is a COMPLETION-class rule (testid-presence /
feature-wiring = the UI is half-built), and finishing it requires ADDING the create/edit/delete
UI, which transiently spikes the count — WS-B's count-only rollback reverted that constructive
work and parked the model (valbuild23: testid-presence -> spray 13 -> undo -> 7 -> undo -> 16 ->
undo -> park at 17). The completion-phase machinery already stands WS-B down; these two rule ids
were just missing from COMPLETION_CLASS_RULES. Also: the rollback now SHOWS the model the new
errors its reverted change introduced (spray delta) instead of hiding them, so it can learn what
to avoid. Full suite green; two new tests. NOT the 'taste rules' framing (that was wrong).
…(article-grab + auth-owner)

Live: valbuild23 crossed the near-green wall to fast-gate green, then PARKED on e2e acceptance —
'Failed to seed a (HTTP 404)'. Root cause: the planner's own worked example teaches every entity
'belongs to a User' (propose-plan.ts:25,62), and parseParents' /^belongs\s*to\s+(\w+)/ captured the
ARTICLE 'a' as the parent entity → a phantom 'aId' FK whose e2e seed POST /api/v1/a 404s. Even
parsed correctly, 'User' is the IMPLICIT auth owner (automatic userId scoping), never a selectable
parent. Fix parseParents: skip a leading article (a/an/the) AND exclude the owner entity (User).
Single source — cascades to the parent select, the e2e seed, and the required-testid list. Tests added.
…rature A/B

DeepSeek V4 Flash docs: temp <0.7 during active thinking collapses CoT diversity → repetitive
reasoning loops. The build runs thinking-ON at DEFAULT_TEMPERATURE=0.2 (the provider sends temp
even with thinking on for deepseek style). Expose a session temperature override (off by default)
so the headless driver can set it via TSFORGE_BUILD_TEMPERATURE for the A/B on the #77 rotation,
without touching the loop. Absent → unchanged (0.2).
… + truthful steer

Both temp-0.2 control builds (A/B) reached fast-gate green then parked
DETERMINISTICALLY on the same wall: the create form submits correctly (row
would appear) but never closes, so the e2e times out waiting for the form to
go hidden (24× resolved to visible <form>). Two compounding causes, two
gate-preserving fixes (the contract is correct — a create form SHOULD close on
success — so teach the model, never relax it):

1. Guide gap (refine-prompt.ts): the form idiom taught submit WIRING
   (void handleSubmit + mutate + onSuccess-invalidate) but never that the form
   must CLOSE on successful create/edit. Front-load the close-on-success idiom:
   the page hook owns showForm/openCreate/closeForm; mutate(input, { onSuccess:
   closeForm }) runs alongside the list-invalidating onSuccess. (Live proof the
   gap is real: control build's own BookmarkPage.hooks.test.tsx built
   openCreate/closeForm/showForm but never wired closeForm into onSuccess.)

2. Contradictory steer (acceptance-steer.ts): the form-hide failure is
   classified 'create', whose generic message says 'ensure the form OPENS' —
   the OPPOSITE of the real fix (it opened fine; it won't CLOSE), misdirecting
   the model to a non-existent open/persist bug. Detect the 'to be hidden' +
   'resolved to visible' signature from the raw detail and steer at closing the
   form on success; fall through to the generic message otherwise.

Tests: form-didn't-close steer emits the close-on-success guidance and NOT the
misleading 'was not visible'; a genuine form-didn't-OPEN failure still uses the
generic create message.
…ctive rename prompt

THE root cause of the bookmarks acceptance wall (live proof valbuild27): the
scaffold ships each entity table with a stub `name` column. A name-less plan
(bookmarks: title+url) makes the model REMOVE `name` and ADD title/url, so
`drizzle-kit push` can't tell rename from drop+create and PROMPTS via
promptColumnsConflicts. In the headless build (no TTY) that throws
"Interactive prompts require a TTY terminal" and exits 1 — and `--force` only
auto-approves data-LOSS, NOT the rename resolver (verified with AND without
--force). The DB never migrates: every query 500s (`column "title" does not
exist`), create fails, the create-form's onSuccess never fires so it never
hides, and acceptance parks on "waiting for <entity>-form to be hidden". Apps
whose plan HAS a `name` field (Company/Contact) keep the stub and only ADD, so
push applies — which is why green was app-shape dependent.

Fix: dbPushForce() runs `db:push -- --force`, and ONLY when it fails with the
interactive-rename signature, drops the (disposable) build entity table over the
same DATABASE_URL connection (the app's own postgres client) and retries — a
clean CREATE that never prompts. Any OTHER push failure is returned unchanged so
a genuinely broken schema (the model's compile error) still surfaces downstream.
Wired at all three push sites (generate, gate command stage, per-feature setup),
each passing the entity's camelCase table name; the gate derives it from
feature.id. Identifier-guarded before it touches SQL.

Tests: db-push.test.ts — success→no drop; rename-prompt→drop+retry; unrelated
failure→not masked; no entityTable→no recover; unsafe identifier→rejected.
Proven manually on valbuild27: DROP + push --force applies title/url cleanly.
…exits 0)

Live proof exposed the real swallow: `bun run db:push -- --force` exits 0 even
when drizzle-kit crashed at the interactive rename prompt — the async rejection
never reaches the exit code (this IS the #60 swallow). The first cut of
dbPushForce gated recovery on `code !== 0`, so it returned early treating the
crash as success and never dropped/retried → DB stayed on the stub `name` column
→ acceptance still parked. Detect the crash by the 'Interactive prompts require a
TTY' / promptColumnsConflicts signature in the captured output regardless of exit
code; recover only from that, pass everything else through unchanged. Tests
updated to model the real code:0 crash.
Panel BLOCK (2 reviewers): the first cut left residual false-green paths — a
swallowed rename crash that recovery COULDN'T fix (drop failed, DATABASE_URL
unset, or the retry still crashed) returned code 0 with the signature, so callers
keying off the exit code read it as success and ran the gate on a stale DB — the
exact #60 false-green the fix targets.

- dbPushForce now NORMALIZES a swallowed crash (code 0 + 'Interactive prompts
  require a TTY' / promptColumnsConflicts signature) to a non-zero code on BOTH
  the no-recovery path and a retry that still crashed, so a failed migration can
  never pass as success.
- dropEntityTable is failure-safe (a throwing/failed drop is swallowed; the retry
  push is the single source of truth).
- boringstackCommandStage SHORT-CIRCUITS on a failed push (returns an actionable
  db-push-failed gate error, raw output included) instead of running the gate
  against a stale schema and reporting green — closes the gate-path false-green.
- Tests: retry-still-crashes → non-zero; no-entityTable/unsafe-ident → surfaced
  non-zero; throwing drop → retry is source of truth; command-stage wiring test
  asserts a persistent rename crash FAILS the stage with the db-push-failed rule.
  Existing gate-stages mock updated (db:push/drop succeed independently of the
  simulated gate outcome). Full suite 3113/0.
…l round-3 coverage)

Panel asked for a dedicated test of generateResource's throw-on-db:push-failure
wiring (was covered only indirectly via dbPushForce). Mock: every command succeeds
except db:push, which emits the headless rename crash on every attempt (exits 0);
dbPushForce cannot recover → surfaces non-zero → generateResource must THROW rather
than proceed to generate:api against a DB that never got the new columns.
Panel: the per-feature setup in boringstackDeps ran its OWN dbPushForce and
IGNORED the result — an untested path that could re-introduce the swallowed
false-green. But it was also redundant: `generate` there IS generateResource,
which already runs the headless-safe dbPushForce (recover-or-throw) every attempt,
and the gate's command stage re-pushes with the same recovery + short-circuit each
cycle. Removed the harness-level push (and the now-unused import); DB-sync
responsibility is solely generateResource's, which surfaces failure by throwing.
Test reframed to assert the delegation (generate is called) AND that the harness
issues no db:push of its own. Full suite green.
…green-completion-class

# Conflicts:
#	packages/core/src/loop/boringstack/gate-stages.ts
#	packages/core/tests/boringstack-gate-stages.test.ts
@agjs
agjs merged commit ba41385 into main Jul 27, 2026
5 checks passed
@agjs
agjs deleted the fix/near-green-completion-class branch July 27, 2026 09:22
agjs added a commit that referenced this pull request Jul 27, 2026
…E2eAcceptance (P0a) (#205)

Observable-gates plan P0a (4-model panel, unanimous). runFinalAcceptance downgraded a
FAILED full gate (validate/build/size/root-drift) to a green 'done' whenever
TSFORGE_NO_E2E_ACCEPTANCE was set — the flag was meant to skip only the browser/chain
e2e, but the terminal-status guard `!finalPassed && !e2eAcceptanceDisabled` also made
the full gate ADVISORY. That is active gate-relaxation and a #204-class false-green (a
real red shipped as done).

finalPassed is false only when (a) the full gate failed or (b) the browser chain failed;
(b) can't occur when the flag is set (the chain is skipped), so when disabled the only way
to !finalPassed is a genuine full-gate red — which must be terminal. Drop the
`&& !e2eAcceptanceDisabled` guard: a failed final acceptance is ALWAYS stuck. The flag
still skips only the chain-acceptance block.

Inverted the test that codified the bug (old 'FIX 5': disabled preserves done on gate
fail) into two: disabled + full-gate-fail → stuck; disabled + full-gate-pass → done with
the chain skipped. Full suite 3140/0.
agjs added a commit that referenced this pull request Jul 27, 2026
… exist after db:push (P0b) (#206)

Observable-gates plan P0b (4-model panel, unanimous #1 control). A green db:push
proves the command RAN, not that Postgres holds the plan's schema (#200/#204: a
swallowed crash exits 0, the DB never migrates, every runtime query 500s while the
gate false-greens). New db-oracle.ts queries information_schema after a successful
push and asserts the plan's columns are physically present — the observable end
state, not the migration's story.

- Expected columns derived from the PLAN (IEntityAcceptance.fields, which already
  includes parent-FK fields) + the scaffold set {id,user_id,created_at,updated_at};
  NEVER from the model's schema file.
- SUBSET-presence (never fail on extras); PRESENCE only (SQL types too coarse);
  name matching camel/snake-tolerant (Drizzle emits user_id; domain cols keep the
  field name). belongs-to-User → the existing user_id.
- INCONCLUSIVE reads (DB unreachable/query error) NEVER block — a read failure right
  after a successful push is transient infra, and final acceptance is the backstop;
  only a successful read with a missing plan column reds. Identifier-guarded table.
- Wired into boringstackCommandStage after the db:push success check; surfaces a
  db-schema-mismatch gate error pointing at app.schema.ts (short-circuits before the
  gate runs on a stale schema).

Proven: 8 oracle unit tests (subset, camel/snake, inconclusive-non-blocking, unsafe
table, missing-column) + a command-stage wiring test (push exit-0 + missing url →
db-schema-mismatch) + LIVE integration against valbuild27's Postgres (real
bookmark{title,url} passes; expecting a missing column reds). Full suite 3149/0.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant