Repository navigation
Replace in-place writes before XNNPACK quantizer annotation - #24499
msluszniak wants to merge 3 commits into
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24499
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New FailureAs of commit 1a3b944 with merge base 5c81773 ( NEW FAILURE - The following job has failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@pytorchbot label "release notes: quantization" |
|
@msluszniak can you split this into two PRs: one for XNNPACK quantizer and one for QNN quantizer. For XNNPACK, tag @JakeStevens for review. For QNN, tag @psiddh for review. |
|
@claude review effort=max |
Reviewing PR #24499
|
prepare_pt2e observes a tensor before a later in-place write through a view, so the quantized model ignores what was written (torchvision's batch_images fills new_full with copy_). Add a replace_in_place_ops transform that rewrites such writes to their functional variants, and call it from XNNPACKQuantizer.transform_for_annotation. Writes to graph inputs and buffers are kept.
|
@JakeStevens claude review didn't finished. Please re-trigger it. |
8c79942 to
e3c44dc
Compare
|
@claude review effort=max |
|
Claude finished @JakeStevens's task in 49m 43s —— View job PR Review: #24499Request changes: the new Detailed reviewSummaryThis PR adds
Code QualityA. Blocker: a second write through a rewritten view is lost ( After a write is rewritten, later users are redirected:
None of these alias the new base. A later in-place op through the same view therefore works on a detached tensor, and so does a later read of the view after another write to the base. The alias guard at :81-87 can't catch this, because it skips buf = x.new_zeros(2, 3)
row = buf.select(0, 0)
row.copy_(x)
row.mul_(2)
return buf + 0 # eager: row 0 == 2 * x; after the pass: row 0 == x
B. Major: aliases from The walk at :78 only follows users that are
C. Minor:
D. Minor:
E. Minor: the new nodes carry stale metadata (
Backward CompatibilityF. Direct in-place ops are rewritten too, although they don't have the bug (
API DesignG. The docstring overstates what is fixed ( These writes are left in place without any diagnostic, and they still hit ao#5040:
A caller such as the planned QNN follow-up can't tell any of this from the docstring. Fix: list the supported views and the cases that are kept, and say H. The new shared pass doesn't follow the
Testing
Specific Comments
RecommendationRequest Changes
Progress
The shallow checkout shares no history with |
A write was rewritten even when the written view, or the in-place op's result, was used again afterwards, so a second write or read through it saw the old value. The alias check also missed unbind/split getitems. Keep those writes, and leave direct in-place ops alone, since later reads already use the in-place node. Rewrite only copy_, fill_, zero_ and arithmetic writes through select, slice and diagonal, compute val for the new nodes under the fake mode, and copy only provenance meta. Make the transform a PassBase, ReplaceInplaceOpsPass, and log the writes it keeps.
|
@JakeStevens addressed the bot review in 3bea937:
|
|
@metascroy split done: this PR is XNNPACK only (@JakeStevens), and the QNN part is #25202 (@psiddh), which depends on it. |
|
@JakeStevens has imported this pull request. If you are a Meta employee, you can view this in D132899587. |
|
The BUCK changes fail internally. The issue is that the pytorch testing import brings in a bunch more dependencies that are not set up in the build. We don't actually need to use that. instead, can you please:
ExecuTorch’s |
torch.testing's common_utils pulls in dependencies the internal Buck build lacks.
|
@JakeStevens done in 1a3b944. |
|
ok internal CI + my checks there look good now too, running claude one last time |
|
@claude review |
|
Claude finished @JakeStevens's task in 6m 13s —— View job PR Review: #24499Approve: the earlier blockers (A, B) and the other follow-ups are fixed, and none of the three reviewers found a new correctness bug. What's left is four minor points: one narrow quantization side effect, writes that are still skipped without any visible message, a few gaps in the docstring, and some missing tests. Detailed reviewSummaryThis PR adds Three independent reviewers covered all six changed files, and I re-read the cited code myself. Nothing was executed, because Python isn't available in this runner. Every case the reviewers traced either rewrites correctly or keeps the original write: the scatter order, the Code Quality
API Design
Testing
RecommendationApprove Nothing blocks merging. The scalar-lifting point is the one I'd fix before landing, or explicitly accept. The rest can be follow-ups. Progress
The diff was taken against |
|
@msluszniak is the quant thru a view an issue for your usecase? |

Summary
prepare_pt2eobserves a tensor before a later in-place write through a view, so the quantized model ignores what was written (pytorch/ao#5040, e.g. torchvision'sbatch_images). Per review in pytorch/ao#5046 this belongs in the quantizer:XNNPACKQuantizernow replaces such writes with functional ops intransform_for_annotation. Only writes through select/slice/diagonal views that stay equivalent are rewritten; writes to inputs and buffers, and the other cases listed in the pass docstring, are kept. The QNN wiring is in #25202.Test plan
New
test_replace_inplace_ops.pyandtest_new_tensor_filled_in_place_through_view(both fail on main). The xnnpack quantizer and transforms suites fail the same tests as main.cc @kimishpatel @jerryzh168 @metascroy @JakeStevens @nil-is-all