[osabnis]

Overview

GLM-4.7-Flash (deepseek2 architecture, MLA) fails on the OpenVINO GPU backend in stateless mode with -fa on/off/auto.

Its per-head MLA weights (attn_k_b and attn_v_b) are 3D quantized tensors, which were rejected by the GPU support check. As a result, their MUL_MAT operations fell back to the CPU (1692 CPU ops with -fa off), splitting the graph at every layer. GPU execution then failed with a shape mismatch between the model input [1,2,20,512] and the tensor shape [1,20,2,512].

This PR enables GPU execution for these weights and addresses the following issues:

• 3D quantized weights: Allow MUL_MAT with a 3D quantized weight when there is one weight matrix per activation batch. The existing translation already handles this as a batched MatMul.
• Exact zero points: Restrict exact zero points to MoE expert weights and limit the GGML_OPENVINO_EXACT_ZERO_POINT opt-in to 2D weights. MLA's 3D weights otherwise use an F16 zero point, which fails in the GPU plugin with CL_OUT_OF_RESOURCES.
• Dynamic VIEW dimensions: Fix dimension selection for k_pe and views of the KV cache.
• KV cache V view: Translate the V view of the K cache as a reshape while preserving the dynamic cell axis, which otherwise spans the entire context.
• Flash Attention patterns: Accept the MLA Q and V patterns in the Flash Attention support check.

Static-shape limitation: The GPU plugin currently fails on MatMul with batched compressed weights when shapes are static. Dynamic shapes work correctly. Since op tests and GGML_OPENVINO_FORCE_STATIC use static shapes, batched quantized weights remain unsupported in those paths.

Additional information

Tested on Arc Pro B70 with OpenVINO 2026.4.1 using a 1024-token prompt, -c 1382 -n 128, stateless:

Flash Attention	Before	After: Prefill / Decode (tok/s)
-fa on	set_input shape error	215.4 / 51.0
-fa off	set_input shape error	213.6 / 51.6
-fa auto	set_input shape error	213.2 / 52.0

• Accuracy: KLD against the CPU backend over 4 wikitext-2 slices with context length 1024: 0.064 / 0.061 / 0.063 for -fa on/off/auto. The top token matches the CPU backend 91% of the time in all three modes. Results are consistent across batch sizes (-ub 512/32/1).
• Regression: Llama-3.2-1B, SmolLM2-1.7B, Phi-3.5-mini, Gemma-4-E2B, and Qwen1.5-MoE tested with -fa on/off/auto: identical output and no new CPU ops. Rechecked on Arc B580 on the current dev_backend_openvino tip (Llama-3.2-1B, Gemma-4-E2B): identical output.
• Operator tests: test-backend-ops -o MUL_MAT on GPU shows no change from the baseline (982/982 on Arc B580).
• Architecture tests: test-llama-archs -a deepseek2 now executes instead of crashing. However, it fails NMSE comparison against the CPU backend, consistent with the existing behavior of MoE architectures on the OpenVINO GPU backend (deepseek, glm4moe, bailingmoe2, and qwen2moe).

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES - AI assistance was used for analysis and running tests.