Repository navigation
Commit 054de1f
authored
Fix crashes, wrong outputs and lowering failures in several backends and in exir (#24222)
Fixes found while exporting models to ExecuTorch backends.
## Fixes
- **exir:** `constant_prop_pass` keeps tensors that in-place ops write,
also through views or in `no_grad`/`autocast` blocks. `to_backend` keeps
the output node last. `to_edge` lowers `torch._check_tensor_all` and
`torch.sym_ite`. `GroupBasedPartitioner` no longer makes partitions that
depend on each other, unless caller node groups already do. Tensor
constants get static shapes; deduplicated constants keep their data.
- **AOTI (AOTInductor) and CUDA:** the missing-kernel check catches
scatter and index_put fallbacks. `detach` lowers, also inside
`torch.cond`. Strided outputs are copied. Transposed convolutions (1-D,
2-D, no dilation) and `torch._check_tensor_all` lower.
- **XNNPACK:** the quantizer skips bfloat16 tensors, and `cat` respects
scoped configs. ELU is delegated when XNNPACK can run it.
- **Qualcomm:** autocast regions are inlined correctly. Unquantized
bfloat16 nodes, data-dependent shapes and HTP partitions without runtime
inputs stay on the CPU. Fixes for stacks, 0-d selects, RMSNorm and int64
constants.
- **OpenVINO:** model data stays alive while OpenVINO 2026.4 reads it.
Partitions are converted with their input types.
- **Core ML:** bfloat16 ops, and ops above rank 5 outside static layout
chains, stay on the CPU.
- **MLX, Arm, Cortex-M, Vulkan, NXP, portable:** dtype checks for MLX
tensor lists, one constant slot per weight, slice updates on a constant
base; TOSA keeps unused delegate inputs for VGF; `MatmulToBmmPass` takes
a constant operand; Vulkan constants above 4 dims; non-strict programs
in the NXP quantizer; `cummin.out`.
## Behavior changes
- A CUDA `.pte` loads only the weights it was exported with. A `.ptd`
with other weights, for example after fine-tuning, needs a newly
exported `.pte`.
- CUDA graphs turn off when outputs do not match the ETensor layout. If
only the capture call sees such a layout, that call returns an error.
- `GroupBasedPartitioner` is about 2x slower on large graphs, and some
graphs get a different number of partitions. `to_backend` keeps the
original order of top-level nodes.
- XNNPACK: weight-only quantized linears, and 8da4w or 8da8w linears
(8-bit dynamic activations, 4-bit or 8-bit weights) lowered with
`to_backend`, fail at export instead of giving wrong results. Delegated
ELU flushes subnormal values to zero. Cats outside a scoped quantizer
config are no longer quantized.
- Ethos-U raises an error for a delegate with unused inputs instead of
returning wrong values.
We would like to cherry-pick this PR to release/1.6 after it merges. It
needs #23915 there first.
## Testing
Each fix has a test that fails without it. Run on an A100 (CUDA,
Vulkan), an x86 CPU (OpenVINO, QNN SDK), macOS (Core ML, MLX), the TOSA
reference model and VGF emulation (Arm), and the eIQ Neutron SDK (NXP).
Not run: Metal, ROCm, Windows CUDA.1 parent 403800d commit 054de1f
85 files changed
Lines changed: 2911 additions & 210 deletions
File tree
- backends
- aoti
- passes
- apple/coreml
- partition
- test
- arm
- ethosu
- test
- misc
- ops
- tosa
- cortex_m
- passes
- test/ops
- cuda
- runtime
- shims
- tests
- test
- tests
- mlx
- builder
- runtime
- test
- nxp
- aten_passes
- quantizer
- tests/generic_tests
- openvino
- runtime
- tests/ops
- qualcomm
- _passes
- partition
- tests
- rework/src
- transforms
- test
- vulkan
- runtime/graph/ops/impl
- test
- xnnpack
- _passes
- operators
- partition/config
- quantizer
- test
- ops
- quantizer
- docs/source/backends/xnnpack
- exir
- backend
- canonical_partitioners
- test
- emit/test
- passes
- program/test
- tests
- verification
- kernels
- aten
- portable
- cpu
- test
- shim_et/xplat/executorch
- build
- kernels/portable
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
10 | 11 | | |
11 | 12 | | |
12 | 13 | | |
| |||
168 | 169 | | |
169 | 170 | | |
170 | 171 | | |
171 | | - | |
| 172 | + | |
| 173 | + | |
172 | 174 | | |
173 | 175 | | |
174 | 176 | | |
| |||
178 | 180 | | |
179 | 181 | | |
180 | 182 | | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
181 | 187 | | |
182 | 188 | | |
183 | 189 | | |
| |||
200 | 206 | | |
201 | 207 | | |
202 | 208 | | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
203 | 229 | | |
204 | 230 | | |
205 | 231 | | |
206 | 232 | | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
207 | 239 | | |
208 | 240 | | |
209 | 241 | | |
| |||
214 | 246 | | |
215 | 247 | | |
216 | 248 | | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
217 | 255 | | |
218 | 256 | | |
219 | 257 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
34 | 40 | | |
35 | 41 | | |
36 | 42 | | |
| |||
40 | 46 | | |
41 | 47 | | |
42 | 48 | | |
43 | | - | |
44 | | - | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
45 | 52 | | |
46 | 53 | | |
47 | | - | |
48 | | - | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
49 | 61 | | |
50 | | - | |
51 | | - | |
| 62 | + | |
| 63 | + | |
52 | 64 | | |
53 | | - | |
54 | | - | |
55 | | - | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
56 | 69 | | |
57 | 70 | | |
58 | 71 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
63 | 63 | | |
64 | 64 | | |
65 | 65 | | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
66 | 125 | | |
67 | 126 | | |
68 | 127 | | |
| |||
83 | 142 | | |
84 | 143 | | |
85 | 144 | | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
86 | 157 | | |
87 | 158 | | |
88 | 159 | | |
| |||
152 | 223 | | |
153 | 224 | | |
154 | 225 | | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
155 | 238 | | |
156 | 239 | | |
157 | 240 | | |
158 | 241 | | |
159 | 242 | | |
160 | 243 | | |
161 | 244 | | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
162 | 252 | | |
163 | 253 | | |
164 | 254 | | |
| |||
362 | 452 | | |
363 | 453 | | |
364 | 454 | | |
| 455 | + | |
365 | 456 | | |
366 | 457 | | |
367 | 458 | | |
368 | 459 | | |
369 | 460 | | |
370 | | - | |
371 | | - | |
372 | | - | |
373 | | - | |
374 | | - | |
375 | | - | |
| 461 | + | |
376 | 462 | | |
377 | 463 | | |
378 | 464 | | |
379 | 465 | | |
380 | 466 | | |
381 | 467 | | |
382 | | - | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
| 476 | + | |
| 477 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
386 | 386 | | |
387 | 387 | | |
388 | 388 | | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
389 | 444 | | |
390 | 445 | | |
391 | 446 | | |
| |||
433 | 488 | | |
434 | 489 | | |
435 | 490 | | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
436 | 524 | | |
437 | 525 | | |
438 | 526 | | |
| |||
0 commit comments