Skip to content

Commit 5779ce9

Browse files
Lijiachen1018lijiachen19
andauthored
[fix] fix sparse attention (#397)
fix ascend attention Co-authored-by: lijiachen19 <[email protected]>
1 parent 8b443e5 commit 5779ce9

File tree

1 file changed

+7
-0
lines changed

1 file changed

+7
-0
lines changed

ucm/integration/vllm/patch/patch_funcs/v092/vllm_ascend_adapt.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,13 @@ def maybe_execute_sparse_attention_finished(
112112
):
113113
if not has_ucm_sparse():
114114
return
115+
ucm_sparse = get_ucm_sparse()
116+
attn_metadata = forward_context.attn_metadata
117+
if attn_metadata is None:
118+
return
119+
ucm_sparse.attention_finished(
120+
query, key, value, attn_output, layer_name, forward_context
121+
)
115122

116123
attention_v1.maybe_execute_sparse_attention_finished = (
117124
maybe_execute_sparse_attention_finished

0 commit comments

Comments
 (0)