[Relax] Return values a match_cast reads from a CUDA graph capture - #20485
Merged
Merged
Conversation
RewriteCUDAGraph lifts static bindings into a capture function and adds a lifted value to the function's outputs when a later binding outside the region uses it. That bookkeeping ran for var, call, tuple and tuple_get_item bindings but not for match_cast, which the planner left to the base visitor. A match_cast over a lifted value therefore stayed in the original function with its input undefined, and VM codegen failed with "Var lv383 is not defined" when compiling Phi-3.5-vision's image embedding for CUDA in MLC LLM. Mark the value as a region output in a MatchCast visitor and add a test.
tlopex
approved these changes
Sep 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RewriteCUDAGraph adds a lifted value to the capture function's outputs when a binding outside the region uses it. The planner does that for var, call, tuple and tuple_get_item bindings but never overrode the match_cast visitor, so a match_cast over a lifted value kept reading a var that no longer exists in the original function. Compiling Phi-3.5-vision for CUDA in MLC LLM hits this in the image embedding, where a static reshape is captured and then match_cast to the symbolic crop grid, and VM codegen fails with
Var lv383 is not defined.MatchCastNodevisitor to the planner that marks the value as an output of the region that lifted it