-
Notifications
You must be signed in to change notification settings - Fork 91
Enable HPU Fused SDPA for Qwen3-VL vision attention using attention masks #787
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
🚧 CI BlockedThe main CI workflow was not started for the following reason:
|
Signed-off-by: slokesha <[email protected]>
c245eb8 to
e07204d
Compare
🚧 CI BlockedThe main CI workflow was not started for the following reason:
|
|
Performance: Main - |
🚧 CI BlockedThe main CI workflow was not started for the following reason:
|
Signed-off-by: Spurthi Lokeshappa <[email protected]>
🚧 CI BlockedThe main CI workflow was not started for the following reason:
|
Qwen3-VL vision attention is updated to use FusedSDPA.apply directly when the query sequence length is within the supported fused range (q_len ≤ 65536).
This removes the per-block Q/K/V attention loop and enables the optimized HPU fused SDPA kernel for vision attention.
The change aligns Qwen3-VL with the optimized path already used by Qwen2.5-VL on Gaudi, improving efficiency while preserving identical model outputs.