Repository navigation
Conversation
…port (pytorch#25183) Summary: Fusion decides which ops to fuse from the quantizer's annotations, but torch.export does not carry quantization_annotation over to the exported graph: it copies only a fixed set of node metadata keys, and the annotation holds references to nodes of the original graph. Fusion also resolved per-channel qparams as get_attr constants, which export lifts to placeholders. So fusion could only run on the converted graph module, before export. annotate_input_qspec_map and annotate_output_qspec in the fused_quant quantizer now wrap the torchao helpers and, when the qspec is live, also set QUANTIZED_KEY in node.meta["custom"], which export keeps. OpQuantizer.fuse gates on that flag instead of the annotation, so it gives the same answer on the converted graph and on a program exported from it. The per-channel qparams path in fuse_aten reads shapes from meta["val"] instead of resolving get_attr tensors, and an affine dequantize's result dtype now comes from its meta["val"] instead of its output_dtype argument (the quantized_decomposed per-tensor and per-channel dequantize ops still read out_dtype), so both also work on an exported program. Differential Revision: D132375392
Summary: A model that returns one of its inputs unchanged is not a valid program to compile. sas_compiler rejected it in `prepare_fn`, which `MODAI_FQ_ET_RECIPE` (D130047758) does not run because ModAI prepares the model itself. This moves that check into the pass manager both entry points run. It supersedes D132346962. `ExportedProgramPassManager` now takes `checks`: functions of the `ExportedProgram` that `check()` runs after `validate()`, so they run before the first pass and after the last. A torch `Verifier` could not express this check: `Verifier.check` is final and its only extension point, `check_additional`, sees the graph module without the graph signature, so it cannot tell a user output from a buffer mutation output or a user input from a lifted parameter. `verify_no_inputs_returned_as_outputs` is the check that was in `prepare_fn`, unchanged: it rejects a `USER_OUTPUT` that names a `USER_INPUT`. A program that writes an input into a buffer, whose buffer mutation output is that input, is still accepted. `FusedQuantPassManager`, the first pass manager after `to_edge` in `compile()`, runs it. The ModAI recipe's `edge_transform_passes` now returns an `ExportedProgramPassManager` with the check instead of a flat pass list. ExportSession hands an `fx.PassManager`, which `ExportedProgramPassManager` subclasses, to `to_edge_transform_and_lower` unflattened, so the check runs there too. Differential Revision: D133544179
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/25361
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (1 Unrelated Failure)As of commit db35d25 with merge base ba0d826 ( BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@rezaasjd has exported this pull request. If you are a Meta employee, you can view the originating Diff in D133544179. |
This PR needs a
|
Summary:
A model that returns one of its inputs unchanged is not a valid program to compile. sas_compiler rejected it in
prepare_fn, whichMODAI_FQ_ET_RECIPE(D130047758) does not run because ModAI prepares the model itself. This moves that check into the pass manager both entry points run. It supersedes D132346962.ExportedProgramPassManagernow takeschecks: functions of theExportedProgramthatcheck()runs aftervalidate(), so they run before the first pass and after the last. A torchVerifiercould not express this check:Verifier.checkis final and its only extension point,check_additional, sees the graph module without the graph signature, so it cannot tell a user output from a buffer mutation output or a user input from a lifted parameter.verify_no_inputs_returned_as_outputsis the check that was inprepare_fn, unchanged: it rejects aUSER_OUTPUTthat names aUSER_INPUT. A program that writes an input into a buffer, whose buffer mutation output is that input, is still accepted.FusedQuantPassManager, the first pass manager afterto_edgeincompile(), runs it. The ModAI recipe'sedge_transform_passesnow returns anExportedProgramPassManagerwith the check instead of a flat pass list. ExportSession hands anfx.PassManager, whichExportedProgramPassManagersubclasses, toto_edge_transform_and_lowerunflattened, so the check runs there too.Differential Revision: D133544179