fix(e2e): reset per test in excalidraw.spec - #7
Merged
Merged
Conversation
The two "flaky" excalidraw specs were deterministic failures.
All three tests bootstrap through /setup, but the file used
test.beforeAll(reset) -- a single TRUNCATE for the whole file. POST
/auth/setup returns 410 (auth.setup_closed) once any user exists, and
SetupPage renders that error WITHOUT navigating, so tests 2 and 3 sat on
/setup until the 30s timeout:
Test timeout of 30000ms exceeded.
Error: locator.click: Test ended.
Call log: - waiting for getByTestId('new-doc')
They passed on retry only because a retry restarts the worker, which
re-runs beforeAll and truncates again. Hence the signature shape: 30s on
the first attempt, 2-7s on retry, and always exactly 2 flaky -- the number
of tests after the first.
The repo already knew about this: change-password and invite-password call
reset() inline in their second test, and create-placement, new-doc-edit-mode
and tree-reorder all use beforeEach. excalidraw was the only multi-test file
that calls /setup in every test without re-resetting.
Each of the three tests is self-contained (own /setup, distinct email), so
resetting per test is safe.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The two "flaky" excalidraw specs were deterministic failures, not flakes.
Cause
All three tests bootstrap through
/setup, but the file usedtest.beforeAll(reset)— one TRUNCATE for the whole file.POST /auth/setupreturns 410 (auth.setup_closed) once any user exists (setup.rs:79), andSetupPagerenders that error and returns without navigating (SetupPage.tsx:22-31).So tests 2 and 3 sat on
/setupuntil the timeout:They passed on retry only because a retry restarts the worker, which re-runs
beforeAlland truncates again. That explains the signature shape exactly:mainin JuneFix
beforeEach(reset). Each test is self-contained (own/setup, distinct email), so per-test reset is safe.The repo already knew about this hazard:
change-passwordandinvite-passwordcallreset()inline in their second test, andcreate-placement,new-doc-edit-modeandtree-reorderusebeforeEach.excalidrawwas the only multi-test file calling/setupin every test without re-resetting.Verification
Docker Desktop isn't running locally, and the convergence test needs collab (which needs toxiproxy, broken on Docker Desktop macOS), so CI is the verifier here — and it reproduces this deterministically. A run reporting
0 flakyinstead of2 flakyis conclusive.🤖 Generated with Claude Code