Skip to content

Commit c8e07a2

Browse files
committed
fix(test-par): CRLF-proof the suite list, serialize deadline-sensitive suites
First CI validation round of the parallel runner, three findings: - Windows failed ALL 104 suites with "0 passed, 1 failed" each: the CRT emits --list-suites lines as CRLF, so every dispatched name carried a trailing CR ("arena\r" is an unknown suite). The union guard caught it loudly, exactly as designed. The driver now strips CR when writing the suite list. - The ubuntu-gcc legs had 3 real failures in the cli suite: it spawns subprocesses with fixed deadlines, and a fully saturated 4-core runner starves those deadlines into flakes. Deadline-sensitive suites (cli, subprocess, watcher, incremental, httpd, ui, index_resilience, and the stack_overflow family) now run SEQUENTIALLY after the parallel wave on a quiet machine — same suites, same tests, same gates, only the schedule differs; the union guard checks the combined result set. - Failing suites now print every FAIL site with context (the tail-30 of a long suite log hid which tests actually failed on CI). Local: totals unchanged (6,361 passed / 0 failed / 1 skipped), union guard clean. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
1 parent bd939fc commit c8e07a2

1 file changed

Lines changed: 40 additions & 5 deletions

File tree

scripts/run-tests-parallel.sh

Lines changed: 40 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,10 @@ mkdir -p "$LOGDIR"
4040
SUITES_FILE="$LOGDIR/suites.txt"
4141
RESULTS_FILE="$LOGDIR/results.txt"
4242

43-
if ! "$RUNNER" --list-suites > "$SUITES_FILE"; then
43+
# tr strips the CR that the Windows CRT appends to every stdout line — a
44+
# suites file with CRLF endings made the runner reject every name
45+
# ("arena\r" is an unknown suite) and fail all 104 suites on CI.
46+
if ! "$RUNNER" --list-suites | tr -d '\r' > "$SUITES_FILE"; then
4447
echo "FAIL: test-runner --list-suites exited nonzero" >&2
4548
exit 1
4649
fi
@@ -49,7 +52,33 @@ if [ "$NSUITES" -lt 1 ] || grep -qvE '^[a-z0-9_]+$' "$SUITES_FILE"; then
4952
echo "FAIL: suite list empty or malformed (runner too old for --list-suites?)" >&2
5053
exit 1
5154
fi
52-
echo "=== parallel test run: $NSUITES suites, $JOBS jobs ==="
55+
# Timing-sensitive suites run SEQUENTIALLY after the parallel wave: they
56+
# spawn subprocesses / watch the filesystem / bind ports with fixed
57+
# deadlines, and a saturated 4-core CI runner starves those deadlines into
58+
# flakes (3 cli-suite failures on the ubuntu legs of the first CI run).
59+
# Same suites, same tests, same gates — only the schedule differs; the
60+
# union guard below still checks the COMBINED result set.
61+
# stack_overflow_a/b/c: their giant-recursion ASan allocations stall ~100x
62+
# when co-STARTED with a large wave on Apple Silicon (2s staggered vs ~230s
63+
# simultaneous — a local scheduler/zone quirk, not contention: job count
64+
# does not change it). Staggered in the tail they cost seconds.
65+
SERIAL_SUITES="cli subprocess watcher incremental httpd ui index_resilience \
66+
stack_overflow_a stack_overflow_b stack_overflow_c"
67+
is_serial() {
68+
case " $SERIAL_SUITES " in *" $1 "*) return 0 ;; *) return 1 ;; esac
69+
}
70+
PAR_FILE="$LOGDIR/suites-parallel.txt"
71+
SER_FILE="$LOGDIR/suites-serial.txt"
72+
: > "$PAR_FILE"
73+
: > "$SER_FILE"
74+
while IFS= read -r sname; do
75+
if is_serial "$sname"; then
76+
echo "$sname" >> "$SER_FILE"
77+
else
78+
echo "$sname" >> "$PAR_FILE"
79+
fi
80+
done < "$SUITES_FILE"
81+
echo "=== parallel test run: $NSUITES suites ($(wc -l < "$SER_FILE" | tr -d ' ') serial-tail), $JOBS jobs ==="
5382

5483
export RUNNER LOGDIR RESULTS_FILE
5584
run_one() {
@@ -67,7 +96,11 @@ run_one() {
6796
}
6897
export -f run_one
6998

70-
xargs -P "$JOBS" -I{} bash -c 'run_one "$@"' _ {} < "$SUITES_FILE"
99+
xargs -P "$JOBS" -I{} bash -c 'run_one "$@"' _ {} < "$PAR_FILE"
100+
# Serial tail: quiet machine for the deadline-sensitive suites.
101+
while IFS= read -r sname; do
102+
run_one "$sname"
103+
done < "$SER_FILE"
71104

72105
# ── Union guard: every listed suite produced exactly one result ──
73106
MISSING=$(comm -23 <(sort "$SUITES_FILE") <(awk '{print $1}' "$RESULTS_FILE" | sort -u))
@@ -88,8 +121,10 @@ echo "── 8 slowest suites ──"
88121
sort -t= -k6 -rn "$RESULTS_FILE" | head -8
89122
grep -v ' rc=0 ' "$RESULTS_FILE" || true
90123
for f in $(grep -v ' rc=0 ' "$RESULTS_FILE" | awk '{print $1}'); do
91-
echo "──── $f (last 30 lines) ────"
92-
tail -30 "$LOGDIR/$f.log"
124+
echo "──── $f: every failure site ────"
125+
grep -B2 -A8 "FAIL" "$LOGDIR/$f.log" | head -120
126+
echo "──── $f: last 15 lines ────"
127+
tail -15 "$LOGDIR/$f.log"
93128
done
94129

95130
echo "────────────────────────────────────────────"

0 commit comments

Comments
 (0)