Skip to content

Commit 722e5cf

Browse files
Fix two crashes caused by constant_prop_pass (#24224)
## What is wrong today Programs that go through `constant_prop_pass` can crash in two ways: one inside the pass, and one later when the program is lowered. **1. A folded value that is not a tensor.** The pass also folds values such as the int from `.item()` on a buffer, and the bools that `torch._check` turns into runtime checks. When a node that stays in the graph reads such a value, the pass tries to turn it into a constant tensor input and fails: ```python n = self.n.item() # self.n is a buffer torch._check(n >= 1) torch._check(n <= x.shape[0]) return x[:n] * self.w[:n] ``` ``` TypeError: cannot create weak reference to 'bool' object ``` **2. A folded parameter that still requires grad.** Folding `self.w.t()`, or one chunk of a packed weight, gives a view that still has `requires_grad=True`, because `torch.no_grad()` does not clear that flag on a view. For some views, such as a transposed one, `to_edge` then copies the constant into a non-leaf tensor, and lowering to a backend can fail when it deep-copies the program: ``` RuntimeError: Only Tensors created explicitly by the user (graph leaves) support the deepcopy protocol at the moment. ``` An op can also return the parameter itself, like `self.w.to(torch.float32)` on a float32 parameter. Then the pass stores the `nn.Parameter` as a constant, and `to_edge` fails with `AttributeError: ... is not a buffer`. ## What this change does Two commits, one per crash: 1. Only tensors can become constant inputs. So a non-tensor value that a node outside the fold reads now stays in the graph, together with the non-tensor values it is computed from. Tensors computed from it still fold: `self.w[:n]` still becomes one constant. Programs that `constant_prop_pass` handled before are unchanged. 2. A folded constant must be a plain tensor that does not require grad, so the pass now detaches every folded tensor. ## What was tested On macOS arm64: - Two new tests in `exir/tests/test_passes.py`, one per commit. Each fails before its commit and passes after: the first with `TypeError: cannot create weak reference to 'int' object`, the second with the `AttributeError` above. - The first covers a scalar read at runtime, one returned as an output, one whose first reader is folded, and one read only by folded nodes (it folds away). The program also lowers with `to_edge(...).to_executorch()`. - The second covers the transposed view and the `to` case, and checks that the output matches eager. - The rest of `exir/tests/test_passes.py`, `exir/emit/test/test_emit.py` and `exir/tests/test_quant_fusion_pass.py`. Five tests in those files fail the same way with and without this change, because they need the quantized and custom op libraries. Co-authored-by: Anthony Shoumikhin <anthony@shoumikh.in>
1 parent 363be8c commit 722e5cf

2 files changed

Lines changed: 75 additions & 0 deletions

File tree

‎exir/passes/constant_prop_pass.py‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -182,8 +182,23 @@ def get_propagated_const_tensor_dict(
182182
and 0 in prop_constant_tensor.stride()
183183
):
184184
prop_constant_tensor = prop_constant_tensor.contiguous()
185+
186+
# A folded constant must be a plain tensor that does not require
187+
# grad. no_grad is not enough when the op returns a view of a
188+
# parameter or the parameter itself.
189+
if isinstance(prop_constant_tensor, torch.Tensor):
190+
prop_constant_tensor = prop_constant_tensor.detach()
185191
const_node_to_tensor[node] = prop_constant_tensor
186192

193+
# Only tensors can be lifted into placeholders. A non-tensor value read by
194+
# a node outside the fold stays a runtime op, together with the non-tensor
195+
# values it is computed from.
196+
for node in reversed(list(const_node_to_tensor)):
197+
if not isinstance(const_node_to_tensor[node], torch.Tensor) and any(
198+
user not in const_node_to_tensor for user in node.users
199+
):
200+
del const_node_to_tensor[node]
201+
187202
return const_node_to_tensor
188203

189204

‎exir/tests/test_passes.py‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2902,6 +2902,66 @@ def forward(self, x: torch.Tensor) -> torch.Tensor:
29022902
module = new_ep.module()
29032903
self.assertFalse(torch.equal(module(x), module(x)))
29042904

2905+
def test_constant_prop_pass_detaches_folded_parameters(self) -> None:
2906+
class M(torch.nn.Module):
2907+
def __init__(self):
2908+
super().__init__()
2909+
self.w = torch.nn.Parameter(torch.randn(4, 4))
2910+
2911+
def forward(self, x):
2912+
# `to` with the same dtype returns the parameter itself.
2913+
return x + self.w.t() + self.w.to(torch.float32)
2914+
2915+
m = M()
2916+
x = torch.randn(4, 4)
2917+
new_ep = constant_prop_pass(export(m, (x,), strict=True))
2918+
2919+
targets = [node.target for node in new_ep.graph.nodes]
2920+
self.assertNotIn(torch.ops.aten.t.default, targets)
2921+
self.assertNotIn(torch.ops.aten.to.dtype, targets)
2922+
self.assertTrue(torch.allclose(new_ep.module()(x), m(x)))
2923+
copy.deepcopy(to_edge(new_ep).exported_program())
2924+
2925+
def test_constant_prop_pass_keeps_scalar_read_at_runtime(self) -> None:
2926+
"""
2927+
A non-tensor value cannot become a constant placeholder. When a node
2928+
outside the fold or the program output reads one, it stays in the
2929+
graph, while the tensors computed from it still fold. A non-tensor
2930+
value read only by folded nodes folds away.
2931+
"""
2932+
2933+
class M(torch.nn.Module):
2934+
def __init__(self):
2935+
super().__init__()
2936+
self.register_buffer("n", torch.tensor(1))
2937+
self.register_buffer("s", torch.tensor(2.0))
2938+
self.register_buffer("k", torch.tensor(3))
2939+
self.w = torch.nn.Parameter(torch.randn(4, 3))
2940+
2941+
def forward(self, x):
2942+
n = (self.n * 2).item() + 1
2943+
torch._check(n >= 1)
2944+
torch._check(n <= x.shape[0])
2945+
# The first reader of `n` is folded, a later one is not.
2946+
w = (self.w * self.s.item())[:n]
2947+
return x[:n] * w, self.k.item()
2948+
2949+
m = M()
2950+
x = torch.randn(4, 3)
2951+
new_ep = constant_prop_pass(export(m, (x,), strict=True))
2952+
2953+
targets = [node.target for node in new_ep.graph.nodes]
2954+
self.assertEqual(targets.count(torch.ops.aten.item.default), 2)
2955+
self.assertIn(torch.ops.aten._assert_scalar.default, targets)
2956+
self.assertEqual(targets.count(torch.ops.aten.mul.Tensor), 1)
2957+
self.assertEqual(targets.count(torch.ops.aten.slice.Tensor), 1)
2958+
self.assertNotIn("s", new_ep.state_dict)
2959+
out, k = new_ep.module()(x)
2960+
expected_out, expected_k = m(x)
2961+
self.assertTrue(torch.allclose(out, expected_out))
2962+
self.assertEqual(k, expected_k)
2963+
to_edge(new_ep).to_executorch()
2964+
29052965
def test_constant_prop_pass_zero_stride_tensors(self) -> None:
29062966
"""
29072967
Test that constant propagation correctly handles tensors with zero strides

0 commit comments

Comments
 (0)