From 94feeec29a4555cdfa9f228f0e8a1ebafd0b7304 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20H=C3=BCning?= Date: Tue, 18 Aug 2026 07:35:45 +0200 Subject: [PATCH 1/2] fix(e2e): reset per test in excalidraw.spec The two "flaky" excalidraw specs were deterministic failures. All three tests bootstrap through /setup, but the file used test.beforeAll(reset) -- a single TRUNCATE for the whole file. POST /auth/setup returns 410 (auth.setup_closed) once any user exists, and SetupPage renders that error WITHOUT navigating, so tests 2 and 3 sat on /setup until the 30s timeout: Test timeout of 30000ms exceeded. Error: locator.click: Test ended. Call log: - waiting for getByTestId('new-doc') They passed on retry only because a retry restarts the worker, which re-runs beforeAll and truncates again. Hence the signature shape: 30s on the first attempt, 2-7s on retry, and always exactly 2 flaky -- the number of tests after the first. The repo already knew about this: change-password and invite-password call reset() inline in their second test, and create-placement, new-doc-edit-mode and tree-reorder all use beforeEach. excalidraw was the only multi-test file that calls /setup in every test without re-resetting. Each of the three tests is self-contained (own /setup, distinct email), so resetting per test is safe. --- e2e/flows/excalidraw.spec.ts | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/e2e/flows/excalidraw.spec.ts b/e2e/flows/excalidraw.spec.ts index 956a5b4..6f69f71 100644 --- a/e2e/flows/excalidraw.spec.ts +++ b/e2e/flows/excalidraw.spec.ts @@ -16,7 +16,14 @@ function reset() { ); } -test.beforeAll(reset); +// beforeEACH, not beforeAll. All three tests below bootstrap through +// /setup, and POST /auth/setup returns 410 (auth.setup_closed) once any user +// exists — SetupPage then renders an error and does NOT navigate, so every +// test after the first hung until the 30s timeout and only passed on retry +// (a retry restarts the worker, which re-runs a beforeAll). That read as +// flakiness for a long time; it was deterministic. Same reason +// create-placement, new-doc-edit-mode and tree-reorder reset per test. +test.beforeEach(reset); /** * Drive a rectangle into the open Excalidraw modal via the dev-only From c1f00959f8ee69d1c7d9e8ccda5cf5d3f445e4e6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20H=C3=BCning?= Date: Tue, 18 Aug 2026 07:45:35 +0200 Subject: [PATCH 2/2] docs: record the excalidraw spec reset fix --- CHANGELOG.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index af841d4..a3bbf20 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,7 +8,16 @@ so this log can be regenerated from history (e.g. with `git-cliff`). ## [Unreleased] -_Changes on `main` that have not yet been tagged._ +### Changed +- `excalidraw.spec` resets per test. Its two long-standing "flaky" specs were + deterministic failures: all three tests bootstrap through `/setup`, but the + file reset once per file with `beforeAll`, and `POST /auth/setup` returns 410 + once a user exists — `SetupPage` renders that error without navigating, so + tests 2 and 3 sat on `/setup` for the full 30s timeout. They passed on retry + only because a retry restarts the worker and re-runs `beforeAll`. That is why + it was always exactly two flaky specs: the number of tests after the first. + The suite now reports `42 passed` with no flakes, and the playwright job + dropped from ~10m to ~5m. ## [0.2.1] - 2026-08-17