fix(io): prevent self-destruction of in-memory durable object alarm task - #7442
Open
spideydotjs wants to merge 1 commit into
Open
spideydotjs wants to merge 1 commit into
spideydotjs wants to merge 1 commit into
Conversation
When an alarm fires in a Durable Object configured with in-memory storage (durableObjectStorage = (inMemory = void)), completing the alarm handler triggers deferred alarm deletion in ActorCache. Under in-memory storage, neverFlush is true, which immediately and synchronously invokes HooksImpl::updateAlarmInMemory(kj::none). Previously, updateAlarmInMemory() unconditionally set maybeAlarmPreviewTask = kj::none. Because runAlarm() was invoked by the coroutine running inside maybeAlarmPreviewTask, clearing the task while it was suspended awaiting runAlarm() destroyed the coroutine frame from within its own callback stack, crashing workerd with "Promise callback destroyed itself". Fix this by tracking whether an alarm execution is currently in flight. When an alarm is running, updateAlarmInMemory() records the updated alarm time without destroying the active task. After runAlarm() finishes, the task loop inspects the scheduled alarm time to determine whether to schedule a subsequent alarm or exit cleanly. Fixes cloudflare#7190, fixes cloudflare#7191
Author
|
I have read the CLA Document and I hereby sign the CLA |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Fixes #7190 and fixes #7191.
When a Durable Object alarm fires under
durableObjectStorage = (inMemory = void), workerd previously terminated with:Root Cause
neverFlushis true inActorCache.ActorCachemarks the alarm deleted and immediately callshooks.updateAlarmInMemory(kj::none)synchronously.HooksImpl::updateAlarmInMemory()was unconditionally resettingmaybeAlarmPreviewTask = kj::none;.runAlarm()was called by the coroutine insidemaybeAlarmPreviewTask. ClearingmaybeAlarmPreviewTaskwhile it was suspended waiting onrunAlarm()caused the coroutine to destroy its own promise node from within its own execution stack, tripping KJ's assertion.Solution
isRunningAlarminHooksImpl.updateAlarmInMemory()stores the updatedscheduledAlarmTimewithout destroyingmaybeAlarmPreviewTask.runAlarm()completes, the coroutine inspectsscheduledAlarmTimeto either schedule the next alarm or exit cleanly.actor-alarms-in-memory-test.wd-testto test in-memory alarm firing and prevent future regressions.