Repository navigation
fix(runtime): stop waiting for stream EOF when exporting runtime files - #96
Merged
JacobLinCool merged 3 commits intoOct 8, 2026
Merged
Conversation
This was referenced Oct 6, 2026
@wasmer/sdk 0.10.0 runs `Command.run()` in a dedicated thread-pool task that sends the exit code and then calls `thread_pool.close()` before the task drops the WASI runner, which owns the stdout/stderr pipe writers. The main-thread scheduler handles `Close` by dropping every `WorkerHandle`, and each drop terminates its Worker. When that termination lands before the task has dropped the runner, stdout and stderr never reach EOF, so `Instance.wait()` (which joins stdout EOF, stderr EOF and the exit code) never settles. The runtime-files stage then idles with no worker threads until the 300 s preparation deadline. Instrumented stalled stages showed all 10,695,683 archive bytes delivered on stdout, both streams still open with a pull pending, and the dedicated worker terminated before it reported idle. CPU contention from concurrent engines widens the window. The WOJFS002 archive is self-delimiting, so read stdout directly and finish once the archive terminator arrives; the parent still verifies the archive SHA-256 before use. A stdout EOF before completion reports stderr. The browser runner Worker used the same `Instance.wait()` export path and now shares the helper. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TakalaWang
force-pushed
the
fix/runtime-preparation-stall
branch
from
October 7, 2026 15:21
dcaba4b to
22b6182
Compare
Contributor
Author
|
@JacobLinCool Rebased onto main after #93 and #95. CI is green and it merges cleanly. Ready for review. Merging it first also clears the unrelated Python stall that failed #99's latest CI run. |
…peScript EOF Reading the runtime-file export until the archive completes left the failure path waiting on stdout EOF, which the same SDK 0.10 teardown race drops. A guest that died mid-archive then sat until the 300 s preparation deadline and lost its traceback. Either stream's EOF means the guest has exited, so the other stream now gets a 2 s idle grace (reset by new bytes) before the read stops and the export fails with stderr. The returned archive is also trimmed to its terminator so chunking cannot change it. TypeScript compilation still awaited Instance.wait() and could stall until the 120 s build timeout (1 of 88 runs at 12x concurrency). It now reads stdout until it holds the driver's one JSON response, then collects stderr until EOF or the same grace. The driver writes that response last, so a complete response stands in for the exit code wait() would report. Both readers share readProcessMessage in src/core/process-output.ts.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Python runtime preparation occasionally stalled until its 300 s deadline when several engines ran at once (about 4 in 100 cold starts downstream in NOJV, with 2 engines per process). The stalled
server-runner-stage.mjschild sat idle inInstance.wait(), with no Wasmer worker threads left.The cause is a race in
@wasmer/sdk0.10.0:Command.run()'s dedicated task sends the exit code.thread_pool.close()before it drops the WASI runner that owns the stdout/stderr pipe writers.Closeby dropping everyWorkerHandle, which terminates its Worker.Instance.wait()never settles.Instrumented stalls showed the complete 10,695,683-byte archive already delivered on stdout, with both streams still open. CPU contention from concurrent engines widens the race window.
Change
Add
readRuntimeFilesExport():WOJFS002archive is complete;The parent still checks the archive SHA-256.
Use it in the server runner stage and in the browser runner Worker, which had the same
Instance.wait()export path. Free the instance afterwards.Verification
New unit test: a complete archive on stdout/stderr streams that never close resolves. A read-to-EOF implementation times out on the same test. An incomplete archive reports stderr.
Repro harness, before → after:
pnpm run typecheck,pnpm run lint,pnpm test(958 passed, 9 skipped) andpnpm run buildall pass.Not covered: the browser path is covered by typecheck and unit tests only, not a real browser run.
Still open: a rarer C++ compile stall seen once downstream did not reproduce here (0 in 160 compiles). The clang stage doesn't use
Instance.wait(), so it may have a different cause.CHANGELOG.mdgets anUnreleasedentry, which may conflict trivially with #93–#95.🤖 Generated with Claude Code