Skip to content

Commit 215b45f

Browse files
committed
Qualcomm AI Engine Direct - Keep float64 tensors on CPU
QNN has no float64 type, so the backend mapped torch.float64 to QNN_DATATYPE_FLOAT_32 and delegated float64 ops. The data was never converted: float64 constants and graph inputs reached QNN as raw 8-byte values read as 4-byte floats. So float64 math gave wrong results with no error. For example, (x.double() * 2 + 1).float() returned zeros on the QNN SDK's x86 simulator for SM8650 in float16 mode, and a sinusoidal timestep embedding computed in float64, as openpi and LeRobot's pi0, pi05 and SmolVLA do, was off by up to 2.0. float64 is now left out of the QNN type map, as bfloat16 already is. A node that reads or writes a float64 tensor stays on CPU, where float64 is computed exactly. Ops on other dtypes are delegated as before. The quantizer's dtype check reads the same map, so it now treats a float64 input as having no QNN type instead of failing on the missing key. On the simulator, both examples above now match PyTorch exactly, and the error of SmolVLA's denoising step drops from 0.32 to 0.0027 (largest absolute error, outputs up to 3.2). New tests lower float64 math between float32 ops, unquantized and quantized, and check the output.
1 parent a521616 commit 215b45f

4 files changed

Lines changed: 19 additions & 3 deletions

File tree

‎backends/qualcomm/builders/node_visitor.py‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -57,8 +57,6 @@
5757
torch.bool: PyQnnManager.Qnn_DataType_t.QNN_DATATYPE_BOOL_8,
5858
torch.float16: PyQnnManager.Qnn_DataType_t.QNN_DATATYPE_FLOAT_16,
5959
torch.float32: PyQnnManager.Qnn_DataType_t.QNN_DATATYPE_FLOAT_32,
60-
# Note that there is no float64 tensor data type in Qnn.
61-
torch.float64: PyQnnManager.Qnn_DataType_t.QNN_DATATYPE_FLOAT_32,
6260
torch.int8: PyQnnManager.Qnn_DataType_t.QNN_DATATYPE_INT_8,
6361
torch.int16: PyQnnManager.Qnn_DataType_t.QNN_DATATYPE_INT_16,
6462
torch.int32: PyQnnManager.Qnn_DataType_t.QNN_DATATYPE_INT_32,

‎backends/qualcomm/quantizer/validators.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,7 @@ def _get_qnn_datatype(node: Node, qspec: QuantizationSpecBase):
104104
):
105105
torch_dtype = torch.uint16 if unsigned else torch.int16
106106

107-
return QNN_QUANT_TYPE_MAP.get(torch_dtype, QNN_TENSOR_TYPE_MAP[torch_dtype])
107+
return QNN_QUANT_TYPE_MAP.get(torch_dtype, QNN_TENSOR_TYPE_MAP.get(torch_dtype))
108108

109109

110110
def _resolve_shared_qspec(

‎backends/qualcomm/tests/models.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1361,6 +1361,11 @@ def forward(self, x):
13611361
return torch.flip(x, self.dims)
13621362

13631363

1364+
class Float64Math(torch.nn.Module):
1365+
def forward(self, x):
1366+
return x + (x.double() * 2 + 1).float()
1367+
1368+
13641369
class Floor(torch.nn.Module):
13651370
def __init__(self):
13661371
super().__init__()

‎backends/qualcomm/tests/test_qnn_delegate.py‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1233,6 +1233,11 @@ def test_qnn_backend_flip(self):
12331233
module = Flip() # noqa: F405
12341234
self.lower_module_and_test_output(module, sample_input)
12351235

1236+
def test_qnn_backend_float64(self):
1237+
module = Float64Math() # noqa: F405
1238+
sample_input = (torch.linspace(0, 1, 8),)
1239+
self.lower_module_and_test_output(module, sample_input)
1240+
12361241
def test_qnn_backend_floor(self):
12371242
sample_input = (torch.randn(3, 4),)
12381243
module = Floor() # noqa: F405
@@ -4447,6 +4452,14 @@ def test_qnn_backend_flip(self):
44474452
module = self.get_qdq_module(module, sample_input)
44484453
self.lower_module_and_test_output(module, sample_input)
44494454

4455+
def test_qnn_backend_float64(self):
4456+
# The default rtol of 1 would pass a float64 result of zeros.
4457+
self.rtol = 1e-1
4458+
module = Float64Math() # noqa: F405
4459+
sample_input = (torch.linspace(0, 1, 8),)
4460+
module = self.get_qdq_module(module, sample_input)
4461+
self.lower_module_and_test_output(module, sample_input)
4462+
44504463
def test_qnn_backend_floor(self):
44514464
sample_input = (torch.randn(3, 4),)
44524465
module = Floor() # noqa: F405

0 commit comments

Comments
 (0)