Repository navigation
Commit 215b45f
committed
Qualcomm AI Engine Direct - Keep float64 tensors on CPU
QNN has no float64 type, so the backend mapped torch.float64 to
QNN_DATATYPE_FLOAT_32 and delegated float64 ops. The data was never
converted: float64 constants and graph inputs reached QNN as raw 8-byte
values read as 4-byte floats. So float64 math gave wrong results with no
error. For example, (x.double() * 2 + 1).float() returned zeros on the
QNN SDK's x86 simulator for SM8650 in float16 mode, and a sinusoidal
timestep embedding computed in float64, as openpi and LeRobot's pi0,
pi05 and SmolVLA do, was off by up to 2.0.
float64 is now left out of the QNN type map, as bfloat16 already is. A
node that reads or writes a float64 tensor stays on CPU, where float64
is computed exactly. Ops on other dtypes are delegated as before. The
quantizer's dtype check reads the same map, so it now treats a float64
input as having no QNN type instead of failing on the missing key.
On the simulator, both examples above now match PyTorch exactly, and
the error of SmolVLA's denoising step drops from 0.32 to 0.0027
(largest absolute error, outputs up to 3.2).
New tests lower float64 math between float32 ops, unquantized and
quantized, and check the output.1 parent a521616 commit 215b45f
4 files changed
Lines changed: 19 additions & 3 deletions
File tree
- backends/qualcomm
- builders
- quantizer
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
60 | | - | |
61 | | - | |
62 | 60 | | |
63 | 61 | | |
64 | 62 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
104 | 104 | | |
105 | 105 | | |
106 | 106 | | |
107 | | - | |
| 107 | + | |
108 | 108 | | |
109 | 109 | | |
110 | 110 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1361 | 1361 | | |
1362 | 1362 | | |
1363 | 1363 | | |
| 1364 | + | |
| 1365 | + | |
| 1366 | + | |
| 1367 | + | |
| 1368 | + | |
1364 | 1369 | | |
1365 | 1370 | | |
1366 | 1371 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1233 | 1233 | | |
1234 | 1234 | | |
1235 | 1235 | | |
| 1236 | + | |
| 1237 | + | |
| 1238 | + | |
| 1239 | + | |
| 1240 | + | |
1236 | 1241 | | |
1237 | 1242 | | |
1238 | 1243 | | |
| |||
4447 | 4452 | | |
4448 | 4453 | | |
4449 | 4454 | | |
| 4455 | + | |
| 4456 | + | |
| 4457 | + | |
| 4458 | + | |
| 4459 | + | |
| 4460 | + | |
| 4461 | + | |
| 4462 | + | |
4450 | 4463 | | |
4451 | 4464 | | |
4452 | 4465 | | |
| |||
0 commit comments