Repository navigation
Conversation
…nswers without one The beta died on 2026-09-03 at 01:49:56 and `https://rehearsal.elenchuslab.dev/` still returned 302. Cloudflare Access serves its login page whether or not anything is behind it, so an invitee would have authenticated, received a one-time PIN, signed in, and only then met a dead origin. Every external signal read healthy. Preflight's five axes all passed: `public_surface` is an operator ATTESTATION that Access is configured, and Access genuinely was up. The origin is a separate fact and now gets measured separately. `origin_reachable` discovers what this machine is actually serving from process environments -- the launcher hands every child ELENCHUS_DB (naming the slug) and ELENCHUS_PORT, the same discovery `forget` uses to refuse deleting a room still being served -- and probes each on LOOPBACK. The probe takes a port and no hostname on purpose: the public hostname is the signal that lied, and one that could reach it would eventually be pointed at it. A non-200 fails too; a process that is up but broken serves the invitee an error. Nothing served is a WARN, never an ok: a machine serving nobody has verified nothing, and "no instances running" must not read like "every instance healthy". THE AXIS FLAPPED WHILE I BUILT IT, AND THE CAUSE IS THE INTERESTING PART. It reported FAIL five times running against a beta the tenant log proved was up throughout, then ok ten times minutes later. `urllib.request.urlopen` consults http_proxy/https_proxy/ALL_PROXY and will route even a 127.0.0.1 request through a proxy; tool invocations inherit those variables inconsistently, so a live origin read as dead. The probe now builds an opener with an EMPTY ProxyHandler -- a loopback health check has no business leaving the machine -- and is pinned by a test that stands up a real server on a real port with every proxy variable set. Verified: 10/10 ok with proxies forced on, and a genuinely dead port still FAILs under the same conditions. It also probes twice before convicting, since one refused connection is weak evidence for an axis that blocks a launch. That retry was written when I still believed the flap was transient; it is kept because it is independently right, not because it was the fix. (Amends the first version of this commit, whose message claimed "five consecutive CLI runs all ok" when the output it cited showed five FAILs. I misread my own terminal.) felix gate pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When the beta died on 2026-09-03,
https://rehearsal.elenchuslab.dev/still returned 302 — Access serves its login page whether or not an origin is behind it. An invitee would authenticate, get a PIN, sign in, and only then meet nothing. All five preflight axes passed, becausepublic_surfaceattests that Access is configured and Access genuinely was up.origin_reachablediscovers served invitees from process environments (ELENCHUS_DBnames the slug,ELENCHUS_PORTthe port — the same discoveryforgetuses) and probes each on loopback. Non-200 fails; nothing-served is a WARN, never an ok.The axis flapped while I built it, and the cause is worth knowing:
urlopenconsultshttp_proxy/ALL_PROXYand routes even 127.0.0.1 through a proxy, so a live origin read as dead five times running, then healthy ten times minutes later. It now uses an emptyProxyHandler, pinned by a test that stands up a real server with every proxy variable set. 10/10 ok with proxies forced on; a dead port still FAILs under the same conditions.Suite 1448 / 35,
felix gatepass. Needs one review.