Repository navigation
Place pt2e input observers after in-place writes through views - #5046
msluszniak wants to merge 2 commits into
Conversation
prepare_pt2e inserted an input observer right after the producer, so a tensor filled later through a view (e.g. buf.select(0, 0).copy_(x)) was observed, and after convert read, before the write. Observers on such a view chain also made the write land on a dequantized copy. Fixes pytorch#5040
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/5046
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
I think to simplify the flow we don't want to support inplace ops, an easier way is probably to change inplace ops to non-inplace first and then run the normal flow (downstream lowering can add inplace ops back if needed), or skip quantizing inplace ops |
Sure, I can rewrite this one to work the way you described it. |
|
@msluszniak can you add this to the quantizer you are using? core pt2e flow probably should not take a side on whether to change inplace ops to non inplace ops or just skip quantizing these |
|
Thanks, moved it into the ExecuTorch XNNPACK and QNN quantizers: pytorch/executorch#24499. Closing this one. |
Fixes #5040.
prepare_pt2enow replaces in-place ops on tensors created in the graph with their functional variants (throughselect/slice/diagonal/as_stridedviews) before annotation. Writes to inputs and buffers are unchanged.Test:
test_quantize_buffer_filled_in_place_through_view(fails on main for both quantizers). pt2e suites pass locally.