Skip to content

[Espressif] Lower Fused Quant linear to ESP-NN - #25153

Open
rascani wants to merge 1 commit into
codex/espressif-fused-quant-linear-quantizerfrom
codex/espressif-fused-quant-linear
Open

rascani wants to merge 1 commit into
codex/espressif-fused-quant-linear-quantizerfrom
codex/espressif-fused-quant-linear

Conversation

@rascani

@rascani rascani commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Lower supported fused_quant.linear calls to ESP-NN's batched per-channel fully connected API. Static contiguous inputs of rank one or higher use the row-based kernel without layout copies, with constant symmetric per-channel int8 weights and optional derived int32 bias. LowerToEsp uses the stored bias and prepares bounded per-channel requantization parameters.

Unsupported calls use shared portable decomposition, including runtime weights or bias, oversized dimensions, partial quantization and unsafe accumulators. Strict export rejects the remaining portable compute. Serialized programs contain neither Fused Quant nor aten.linear operators.

Validation: all 28 host linear cases passed, including the focused portable-report rerun. Shared cases cover accelerated and portable calls, bias/no bias, vectors and higher ranks, aligned/tail widths, shared bias, float users, ReLU, chains, large bias and saturation. The fully composed stack also passed the runtime-bias and runtime-weight cases on the ESP32-S3 emulator. Scoped lint passed.

Authored with OpenAI Codex.

@rascani
rascani added this pull request to stack #25135 November 17, 2026 01:49
@pytorch-bot

pytorch-bot Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/25153

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures

As of commit a7c1599 with merge base 351e938 (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 17, 2026
@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@rascani

rascani commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

@claude review effort=xhigh

@claude

claude Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Claude finished @rascani's task in 4m 3s —— View job


I found no correctness blockers, but every unsupported linear (dynamic batch dimension, runtime bias, pruned channels) now fails the whole export with no fallback, and the C++ kernel doesn't check the shape and dtype relationships between its arguments.

Detailed review

PR Review: #25153

Summary

This PR lowers fused_quant.linear to esp_nn_fully_connected_per_ch_s8_batch. The argument order, offset signs, shift and multiplier conventions, and the bias-scale contract with the quantizer all match across the kernel, operators.py/operators.yaml and the pass. The problems are:

  • The design choice that rejected linears always hard-fail the export.
  • Gaps in the kernel's argument checks.
  • A few smaller issues in the pass.
  • Gaps in the tests for the rejection paths.

The _batch ESP-NN symbol couldn't be checked offline because ESP-NN isn't vendored here. It was only exercised on the ESP32-S3 emulator. Please confirm it exists for the other targets in idf_component.yml, such as ESP32 and P4.

Code Quality

  • Rejected linears hard-fail even with allow_cpu_fallback=True. In backends/espressif/passes/fused_quant_to_esp_pass.py:436-442, any ValueError from _linear_kernel_args becomes a RuntimeError. Every other op in this pass returns None and stays on the portable path. Inputs that now abort the export include:

    • a dynamic batch dimension (next bullet);
    • a runtime float bias;
    • in or out features above 65535;
    • a custom quantizer with per-tensor weights;
    • any single channel whose shift is below -30.

    The PR description says aten.linear has no portable kernel. A more contained fix would be to have the FQ decomposition, or a small ESP pass, emit core ops (permute_copy + mm/addmm) for rejected linears. The pass would then raise only when allow_cpu_fallback=False. If you keep the current behaviour, the README should state clearly that any linear outside the supported envelope blocks export entirely.

  • Dynamic leading dimensions are rejected unnecessarily. fused_quant_to_esp_pass.py:322 (all(type(dim) is int for dim in value.shape)) rejects a symbolic batch dimension. The kernel already computes batches = input.numel() / columns at runtime, so only the last dimension needs to be static.

    • Trigger: export nn.Linear with dynamic_shapes={"x": {0: Dim("b")}}. Because of the previous bullet, this now aborts with no fallback.
    • Supporting it would also need resize_tensor(out, …) in the kernel.
  • One degenerate channel fails the whole model. fused_quant_to_esp_pass.py:275 raises when any channel's shift is below -30.

    • Trigger: a pruned (all-zero) weight row gets the observer's floor scale of about 1.19e-7. That pushes its rescale below 2^-31 whenever input_scale/output_scale is under about 3.9e-3.
    • That channel's output is just bias plus zero point, so clamping the shift to -31 (its contribution rounds to 0 anyway) would be safer than rejecting.
    • Confidence: medium. I traced this from the code; I didn't reproduce it with real scales.
  • The overflow bound subtracts the output zero point in the wrong domain. At fused_quant_to_esp_pass.py:283-284, - abs(output_params.zp) is applied to the pre-requantization accumulator bound. The zero point is added after the int32 rescale, so the bound should be limit << max(shift, 0) <= INT32_MAX. As written it is only over-conservative: it spuriously rejects linears whose bound is within about 128 of the limit.

  • Whole-graph constant propagation just to look up placeholders. fused_quant_to_esp_pass.py:427-433 calls get_propagated_const_tensor_dict and evaluates every foldable node in the program. The weight, scales, zero points and bias it needs are lifted placeholders, which the rest of this pass reads with get_constant.

    • It also silently accepts a computed call_function "constant" as the weight or bias. Example: an unfolded aten.clone for a shared bias. The ESP kernel would then be fed by a portable op the README doesn't mention.
    • Either get_constant is enough (drop the propagation), or the computed node needs to be replaced with a lifted constant.
    • Confidence: low on whether such a node survives into this pass in practice.
  • Minor numerical inconsistency. The bias was quantized with the float32 product input_scale * weight_scale (checked at :390). The output multiplier at :272 is computed in double precision. The relative error is around 1e-7 and probably never visible, but reusing the same derived bias scale for both would make them consistent.

Infrastructure

  • The kernel doesn't check how its arguments relate. Problems in backends/espressif/ops/op_quantized_linear.cpp:34-60:

    • weight.dim() >= 2 should be == 2. A [C, K, X] weight silently ignores its trailing dimension.
    • batches = input.numel() / columns uses integer division without checking input.size(input.dim() - 1) == columns. A remainder is dropped and rows are read from misaligned offsets.
    • Nothing checks out.numel() == batches * channels or bias->numel() == channels. The nbytes checks only stop out-of-bounds access, not shape mismatches.
    • Nothing checks scalar_type(). Input, weight and out are read as int8; bias, multiplier and shift as int32. An int64 shift tensor would be reinterpreted as interleaved int32 values.

    This is the first ESP kernel whose shapes are related across arguments, and the first to take int32 parameter tensors, so these cheap ET_KERNEL_CHECKs are worth adding. All three reviewers flagged this.

Testing

  • Most rejection branches in _linear_kernel_args are untested. backends/espressif/test/ops/test_linear.py:198-263 covers runtime bias, the 65535 limit, a float input or output edge, and overflow. It does not cover:
    • dynamic or symbolic shapes (the most user-visible one, given the hard-fail policy);
    • per-tensor or nonzero weight zero points;
    • bias qparams that aren't derived (torch.equal(bias_scales, …) at :390), which is the likely path for a user-supplied FusedQuantQuantizer with a plain bias qspec;
    • shift below -30;
    • the "float32 reference may overflow" branch.
  • The overflow test doesn't model a realistic overflow. test_accumulator_overflow_reports_export_error (test_linear.py:254) uses all-zero weights and inputs, so it relies on observer floor scales and a saturated int32 bias. Which check fires depends on observer details. A model with real weights and a large but representable bias that overflows only after << shift would test the property the test is named for.
    • Confidence: low. The PR reports this test passing.

Recommendation

Needs Discussion. The main open question is the always-fail policy for rejected linears. Adding the kernel shape and dtype checks, and either allowing a dynamic batch dimension or documenting the restriction, would address the rest.

@rascani

rascani commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

@claude /pr-review

Please review the current revision (857a90b8aab410f01d0a88837cb4eea309f5dcf5) against this PR's base branch, codex/espressif-fused-quant-linear-quantizer. This PR is part of stack #25158. Re-check any previous findings against the updated code and focus on actionable correctness, quantization/numerical, memory-safety, and test-coverage issues.

@claude

claude Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Claude finished @rascani's task in 25m 32s —— View job


At 857a90b I found no correctness blockers on the supported path, but some earlier findings still hold: a linear that can't be lowered still aborts the export even with CPU fallback (this is now documented), float aten.linear left by a filtered quantizer isn't caught, the kernel doesn't check how its argument shapes relate, and the positive-shift and rejection paths have no tests.

Detailed review

PR Review: #25153

Summary

This PR lowers fused_quant.linear to esp_nn_fully_connected_per_ch_s8_batch. LowerToEsp computes the per-channel requantization parameters and raises an export error for any linear outside the supported envelope.

The supported path has no correctness blockers. The latest Test ESP32 Emulators run on this PR passed. That run covers ESP32-S3 with both the optimized and ansi ESP-NN kernels, and executes test_implementation_linear through .ci/scripts/test_espressif.sh:158.

What remains:

  • the no-fallback policy;
  • the kernel's missing checks between related arguments;
  • missing tests for the positive-shift and rejection paths.

Previous review findings, re-checked at this revision:

  • Still present:
    • the no-fallback policy (now documented in README.md:41-42 and lowering.py:133);
    • dynamic batch is rejected;
    • the kernel argument checks;
    • whole-graph constant propagation;
    • the untested rejection branches.
  • Withdrawn:
    • The - abs(output_params.zp) in the overflow bound (lower_to_esp.py:288) is not in the wrong domain. The Q31 rescale never increases magnitude, so the bound also covers adding the zero point after the rescale. _mul_multiplier:47 does the same.
    • The shift < -30 rejection (:280) can't be reached with the default quantizer. XNNPACK's config floors weight scales at eps=2**-12 (backends/xnnpack/quantizer/xnnpack_quantizer.py:154). Triggering it would need s_out > 2**19 * s_in.
    • The float32 bias scale vs double multiplier mismatch is negligible. It is at most 2^-24 relative, far below one int8 step.
    • The ESP-NN _batch symbol question is answered for ESP32-S3 (optimized and ANSI kernels) by the emulator run above.

Code Quality

  • A linear that can't be lowered still aborts export with allow_cpu_fallback=True. This is at backends/espressif/passes/lower_to_esp.py:440-447. It was in the previous review and the behaviour is unchanged. Unsupported add/mul/pool configurations in the same loop stay portable; linear raises instead. These now block the whole export:
    • A dynamic batch dim (Dim("b")), rejected by type(dim) is int at :325.
      • This one can be fixed separately: the kernel already derives batches from input.numel() (op_quantized_linear.cpp:43).
      • Only input.shape[-1] needs to be static, plus a resize_tensor(out, …) in the kernel.
    • Any biased linear from a custom quantizer built the standard FQ way.
      • OpQuantizer(..., other_names=("bias",)) passes the bias through as float with null qparams (frontends/fused_quant/quantizer/quantizer.py:75-76), and :379-389 rejects that.
      • test_linear.py:231-242 builds exactly this quantizer and only avoids the error by using the no_bias case.
    • Other triggers: a runtime or computed bias, per-tensor weight scales (:366 requires shape (out, 1)), features above 65535, and narrow-range activations.
    • Float aten.linear is not guarded. The check only covers fused_quant.linear.
      • A float aten.linear that a NoopQuantizer or node_filter leaves unquantized is still kept by quantizer.preserved_ops() (lowering.py:148,155; quantizer.py:218-219,306-308), and nothing in EspPassManager decomposes it.
      • With fallback enabled, export succeeds and the program then fails at method load, because portable has no linear.out.
      • Medium confidence: I traced this by reading the code, not by running it.
    • The docs disagree. README.md:176-178 still says "Valid unsupported configurations remain portable", which contradicts README.md:41-42.
    • One error message is misleading. The message at :337 ("require constant per-tensor int8 qparams") is also reported for narrow-range activations, which _qparams rejects at :154-155.
    • Suggested fix: portable already has addmm.out, mm.out and permute_copy.out (kernels/portable/functions.yaml:105,680,741).
      • Decompose any remaining aten.linear after DecomposeFusedQuant in EspPassManager, and let unsupported FQ linears fall through to it instead of raising.
      • _require_no_portable_compute already rejects leftovers when allow_cpu_fallback=False.
      • Fix this →
      • Fix the README and error-message wording independently of the policy: Fix this →
  • The kernel checks byte bounds but not how its arguments relate. This is at backends/espressif/ops/op_quantized_linear.cpp:34-60. It was in the previous review and is unchanged.
    • weight.dim() >= 2 should be == 2.
    • Nothing checks input.size(input.dim() - 1) == columns. As a result, batches = input.numel() / columns silently drops a partial row. If numel < columns, out is never written and the kernel still reports success.
    • Nothing checks out.numel() == batches * channels, or that bias, multiplier and shift each have numel() == channels.
    • Nothing checks dtypes: input, weight and out should be Char; bias, multiplier and shift should be Int.
    • Impact: the nbytes checks keep every access in bounds. So this gives silently wrong output, not memory corruption, and only for malformed programs.
    • This is the first ESP kernel whose arguments' shapes depend on each other, so these are cheap ET_KERNEL_CHECKs to add.
    • Fix this →
  • The pass propagates constants through the whole program just to fetch placeholders. This is at lower_to_esp.py:432-436. It was in the previous review, is unchanged, and has low impact.
    • get_propagated_const_tensor_dict evaluates every foldable call_function in the program.
    • It can also return a computed node as the "constant" weight or bias. The kernel would then be fed by a CPU op.
    • get_constant is enough here; _qparams already uses it at :145-146. The PR's own test reads every linear's bias with get_constant (test_linear.py:183).
    • Fix this →

Testing

  • No test exercises a positive requantization shift.
    • In every case the real multiplier s_in * s_w / s_out is well below 1, so all shifts are negative. I estimated this from the test inputs and weights; I did not run the tests.
    • Untested as a result:
      • the << max(shift, 0) part of the overflow bound (lower_to_esp.py:286-291);
      • the left-shift branch of requantize (utils/quantization.py:32);
      • ESP-NN's left-shift path, both on host and on the emulator.
    • test_accumulator_overflow_reports_export_error reaches the bound through a saturated bias (all-zero weights and inputs), not through << shift.
    • Every input also has zero point ≈ −32. The common post-ReLU case is not covered: a non-negative input has zero point −128, i.e. input_offset = +128.
    • Fix: add a high-cancellation case to test_cases (so it also runs on the emulator) and a non-negative-input case. Fix this →
  • Most rejection branches are still untested. This was in the previous review.
    • Only four are covered: runtime bias, dimensions above 65535, a float input or output edge, and bias overflow.
    • Not covered:
      • the bias-contract check (:393-402), which the PR description asks reviewers to look at first;
      • the weight scale shape and zero-point checks (:363-374);
      • dynamic shapes (:325), the trigger users are most likely to hit under the no-fallback policy;
      • the float32-overflow check (:304-309).
    • Fix: a Dim("batch") case and a custom quantizer with a non-derived int32 bias qspec would cover the two most important. Fix this →
  • qtol applies the largest output scale in the graph to every output (tester.py:124-132).
    • In shared_bias, the second input's range is 8× the first's (ramp_tensor(-24, 40) vs (-3, 5)). So the first call's output can be off by several of its own int8 steps and still pass.
    • Those are exactly the per-call bias-scale errors this case is meant to catch. chain has the same issue to a lesser degree.
    • Fix: use a per-output tolerance, or give both inputs the same range. Fix this →

Recommendation

Needs Discussion. The supported path is numerically consistent across the pass, the reference op and the kernel, and passes the ESP32-S3 emulator tests. The open question is still the no-fallback policy for rejected linears. It is now documented, but it also leaves float aten.linear from filtered quantizers unchecked. The kernel checks and the test additions are small.

I did not run any tests: Python and pytest need approval in this runner, so this review comes from reading the code. ESP-NN sources aren't vendored, so the ESP-NN side is verified only through the emulator CI result.

@rascani

rascani commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

@claude /pr-review

Please re-review the current revision (0c4cb18da2a5d0e9cac1f36c8683a707f2ca361c) against this PR's base branch, codex/espressif-fused-quant-linear-quantizer. This PR is part of stack #25158. Re-check previous findings against the updated code and focus on actionable correctness, quantization/numerical, memory-safety, and regression-test issues in this PR.

This PR now enables the linear quantization rule together with the ESP-NN kernel; #25152 only prepares the rule. Please re-check the landing split and public recipe behavior. Unsupported FQ linear fallback remains an explicit existing limitation, while unquantized linear uses the portable decomposition. The focused quantizer and host linear checks passed.

@claude

claude Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Claude finished @rascani's task in 29m 47s —— View job


Re-review of 0c4cb18: no memory-safety or requantization-math blockers, but turning on the linear rule by default makes get_esp_fused_quant_recipe() fail on linear layers that the base branch exported as portable float. A quantizer node_filter would avoid this. A few minor kernel-check and test gaps remain.

Detailed review

PR Review: #25153

Summary

This PR adds _LinearQuantizer to the default Espressif Fused Quant quantizer, lowers fused_quant.linear to the new esp_nn::quantized_linear_int8 kernel (esp_nn_fully_connected_per_ch_s8_batch), and adds host and emulator tests.

  • Landing split: no issues. [Espressif] Prepare Fused Quant linear annotation and int32 bias #25152 adds the rule without registering it; this PR registers it together with the kernel and the lowering.
  • Kernel and math: no memory-safety problem in the kernel. No problem in the requantization, overflow or bias-contract math.
  • CI: the ESP32 emulator run for this commit passed both ESP32-S3 jobs (optimized and ansi). In the optimized job log, all 16 test_implementation_linear cases passed. I did not run tests locally.
  • Main issue: the public recipe now fails on linear layers that it could have left float (see API Design).

Previous findings:

  • Fixed: the README/docstring contradiction.
  • Withdrawn: "float aten.linear not guarded". An unquantized linear does decompose to permute_copy + addmm in TO_EDGE.
  • Dropped: the qtol max-scale note, because test_dialect is not changed in this PR.
  • Still open:
    • the kernel shape checks and the untested branches (below);
    • the misleading "constant per-tensor int8 qparams" error for valid narrow-range qparams (lower_to_esp.py:339);
    • constant propagation over the whole program whenever any linear exists (lower_to_esp.py:434-438, low impact).

API Design

  • The default recipe now fails on linear layers that the base branch exported as portable float (major).

    • At base, _LinearQuantizer was not registered, and test_default_recipe_keeps_linear_portable checked that a float linear became permute_copy + addmm. This PR registers the rule for every linear (esp_fused_quantizer.py:103) and deletes that test.

    • With get_esp_fused_quant_recipe() and its default allow_cpu_fallback=True, these models now fail at export:

      • runtime or computed weights, which raise in the quantizer (esp_fused_quantizer.py:45-46). test_runtime_linear_weights_report_error now asserts this through the default quantizer.
      • a runtime bias (lower_to_esp.py:378-391).
      • any weight dimension above 65535 (:340-351), for example a large-vocabulary output head.
      • export(..., dynamic_shapes=...) (:323-334).
    • Fixing this does not need the deferred FQ-linear fallback. None of these conditions depend on calibration, so the quantizer can detect them up front. It can then leave those layers float, and they take the portable addmm path you describe.

    • Suggested fix: give _LinearQuantizer a node_filter, like _AvgPoolQuantizer at :108. The filter would check that:

      • the weight is a get_attr;
      • the bias is None or a get_attr;
      • both weight dimensions are in [1, 65535];
      • the input and output shapes are static.

      After that, only the checks that depend on calibration would still raise: the shift range and the int32/float32 overflow checks.

    • If you keep the hard failure, the class docstring at esp_fused_quantizer.py:93-94 still says the remaining operators decompose for portable execution. It needs the same linear caveat as README.md:165-167 and lowering.py:133.

    Fix this →

Code Quality

  • The kernel checks buffer sizes but not the linear shape contract (minor). This is not memory-unsafe, because the nbytes checks at op_quantized_linear.cpp:52-60 bound every read and write. The gaps are:

    • weight.dim() >= 2 (:34) accepts weights with more than two dimensions. The lowering only produces 2-D weights, and the pool kernels check for an exact dim().
    • batches = input.numel() / columns (:43) silently misreads the input when its last dimension is not columns.
    • An out or per-channel tensor with the wrong size is not rejected.

    Suggested fix: use weight.dim() == 2, and add checks for:

    • input.size(input.dim() - 1) == columns;
    • out.numel() == batches * channels;
    • numel() == channels for output_multiplier, output_shift and bias.

    Fix this →

Testing

  • No recipe-level regression test for an unquantized linear (minor).

    • test_default_recipe_keeps_linear_portable was deleted and nothing replaced it.
    • test_operator_library_uses_standard_edge_decompositions (test_pass_manager.py:185) calls to_edge_transform_and_lower directly, not the recipe.
    • The recipe differs in one way that matters here: its _core_aten_ops_exception_list=quantizer.preserved_ops() (lowering.py:155) lets aten.linear into edge IR. If an unquantized linear stopped being decomposed, edge IR validation would not catch it, and there is no portable aten::linear kernel.
    • Suggested test: run the recipe on a linear that the quantizer leaves float, using the filter above or a custom quantizer. Assert that the plan contains aten::addmm and no aten::linear.

    Fix this →

  • The positive-shift path and several rejection branches are untested (minor).

    • By my estimate from their weights and input ranges, every case in test_linear.py has s_in·s_w/s_out ≪ 1. So the << max(shift, 0) overflow bound (lower_to_esp.py:289) is never exercised.
    • ESP-NN's left-shift requantization also never runs on the emulator, which is the only place the vendor kernel is compared against the Python reference.
    • Add a case with a multiplier ≥ 1, for example strongly cancelling weights calibrated on a narrow output range.
    • These rejection branches have no tests:
      • shift < −30 (:282);
      • float32 reference overflow (:306-311);
      • the weight scale/zero-point checks (:365-376);
      • the bias-scale contract (:395-404).

    Fix this →

Recommendation

Needs Discussion: the open question is what the default recipe should do with linear layers it can tell before calibration that it cannot lower. It can raise, as now, or leave them float. Those models exported at base, and a node_filter would keep them working until a portable FQ-linear fallback exists. The other items are minor.


  • Gather PR context and diff against codex/espressif-fused-quant-linear-quantizer
  • Run independent reviews
  • Consolidate findings and re-check previous ones
  • Fact-check each finding against the code and CI logs
  • Post the review

Lower supported quantized linear calls to ESP-NN's batched per-channel fully connected API. Contiguous vector and higher-rank inputs use the row-based kernel without layout copies. Reuse the quantizer's int32 bias and prepare constant per-channel multipliers with bounds on integer accumulation and floating reference overflow.

Unsupported calls use shared portable decomposition, including runtime weights or bias, oversized dimensions, partial quantization and unsafe accumulators. Strict export rejects those portable operators. The shared dialect and S3 cases cover accelerated and portable linear calls and serialize without remaining Fused Quant or aten.linear operators.

Authored with OpenAI Codex.

This branch was previously deployed

1 inactive deployment
cadence — a7c15996 Deployed Nov 21, 2026 by rascani via hifi5s-op-test / hifi5s #39863
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants