Repository navigation
Leave a method ready to run again after a kernel throws - #25415
Open
shoumikhin wants to merge 1 commit into
Open
shoumikhin wants to merge 1 commit into
shoumikhin wants to merge 1 commit into
Conversation
Method::execute keeps its progress in the method, and resets it when an
instruction returns an error. When a kernel or a backend throws a C++
exception instead, the exception leaves execute() halfway with the progress
still set. Every later execute() then fails with InvalidState ("Method
execution is in progress") until the method is loaded again. So a caller
that catches the exception, such as a Python binding, cannot run the method
again.
The same exception also skips the reset of the temporary allocator that
follows every instruction. Temporary memory the kernel took before it threw
stays allocated. With the default allocator, which uses malloc, that memory
leaks if the caller destroys the method instead of running it again.
execute() now resets its progress however it ends, and each instruction
resets the temporary allocator however it ends, also when an exception
passes through. Runs that return normally behave as before. Builds without
exception support are not affected.
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/25415
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit bc58034 with merge base ba0d826 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is wrong today
Method::executekeeps its progress in the method and resets it when an instruction returns an error. When a kernel or a backend throws a C++ exception instead, the exception leavesexecute()halfway with the progress still set. If the throw comes after the first instruction, every laterexecute()then fails withInvalidState("Method execution is in progress") until the method is loaded again. So a caller that catches the exception, such as a Python binding, cannot run the method again.malloc, that memory leaks if the caller destroys the method instead of running it again.What this changes
execute()now resets its progress however it ends. Each instruction resets the temporary allocator however it ends, so this also coversMethod::step(). Runs that return normally behave as before. Builds without exception support are not affected.What was tested
KernelIntegrationTest.ThrowingKernelLeavesMethodReadyToRunAgainmakes a kernel throw past the first instruction, then runs the method again.KernelTempMemoryAllocatorIntegrationTest.ThrowResetsTempMemorymakes a kernel take temporary memory and then throw. Both fail without this change and pass with it.