Skip to content

Leave a method ready to run again after a kernel throws - #25415

Open
shoumikhin wants to merge 1 commit into
pytorch:mainfrom
shoumikhin:execute-after-throw
Open

shoumikhin wants to merge 1 commit into
pytorch:mainfrom
shoumikhin:execute-after-throw

Conversation

@shoumikhin

Copy link
Copy Markdown
Contributor

What is wrong today

  • Method::execute keeps its progress in the method and resets it when an instruction returns an error. When a kernel or a backend throws a C++ exception instead, the exception leaves execute() halfway with the progress still set. If the throw comes after the first instruction, every later execute() then fails with InvalidState ("Method execution is in progress") until the method is loaded again. So a caller that catches the exception, such as a Python binding, cannot run the method again.
  • The same exception skips the reset of the temporary allocator that follows every instruction. Temporary memory that the kernel took before it threw stays allocated. With the default allocator, which uses malloc, that memory leaks if the caller destroys the method instead of running it again.

What this changes

execute() now resets its progress however it ends. Each instruction resets the temporary allocator however it ends, so this also covers Method::step(). Runs that return normally behave as before. Builds without exception support are not affected.

What was tested

  • Built and ran the full C++ test suite on Linux x86_64 (CMake, Debug).
  • New tests, built only with exceptions: KernelIntegrationTest.ThrowingKernelLeavesMethodReadyToRunAgain makes a kernel throw past the first instruction, then runs the method again. KernelTempMemoryAllocatorIntegrationTest.ThrowResetsTempMemory makes a kernel take temporary memory and then throw. Both fail without this change and pass with it.

Method::execute keeps its progress in the method, and resets it when an
instruction returns an error. When a kernel or a backend throws a C++
exception instead, the exception leaves execute() halfway with the progress
still set. Every later execute() then fails with InvalidState ("Method
execution is in progress") until the method is loaded again. So a caller
that catches the exception, such as a Python binding, cannot run the method
again.

The same exception also skips the reset of the temporary allocator that
follows every instruction. Temporary memory the kernel took before it threw
stays allocated. With the default allocator, which uses malloc, that memory
leaks if the caller destroys the method instead of running it again.

execute() now resets its progress however it ends, and each instruction
resets the temporary allocator however it ends, also when an exception
passes through. Runs that return normally behave as before. Builds without
exception support are not affected.
@shoumikhin shoumikhin added the release notes: runtime Changes related to the core runtime which loads the program methods, initializes delegates, and runs label Nov 21, 2026
Copilot AI balanced review requested due to automatic review settings November 21, 2026 14:21
@pytorch-bot

pytorch-bot Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/25415

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit bc58034 with merge base ba0d826 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 21, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. release notes: runtime Changes related to the core runtime which loads the program methods, initializes delegates, and runs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants