Skip to content

[Espressif] Fuse linear ReLU and HardTanh patterns - #25178

Open
rascani wants to merge 1 commit into
codex/espressif-fused-quant-activation-fusionfrom
codex/espressif-fused-quant-linear-activation-fusion
Open

rascani wants to merge 1 commit into
codex/espressif-fused-quant-activation-fusionfrom
codex/espressif-fused-quant-linear-activation-fusion

Conversation

@rascani

@rascani rascani commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Fuse single-consumer linear followed by ReLU or HardTanh, including ReLU6, into the ESP-NN fully connected call. Quantization observes the pattern boundaries, and FoldActivations transfers the final output qparams and clamp bounds while preserving per-channel weights and derived int32 bias. Forked and separately quantized activations retain their boundaries.

Review quantization and activation folding before the linear lowering and schema/wrapper updates. Shared dialect and S3 cases cover bias/no bias, forks, saturation, vector and higher-rank inputs, and narrow asymmetric, positive-only, and negative-only clamp grids.

Validation: linear-fusion and quantizer tests pass within the revised stack's 978-test Python suite. Scoped lint passed.

Authored with OpenAI Codex.

@rascani
rascani added this pull request to stack #25158 November 17, 2026 18:10
@pytorch-bot

pytorch-bot Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/25178

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 Cancelled Jobs

As of commit 93263ca with merge base 351e938 (image):

CANCELLED JOBS - The following jobs were cancelled. Please retry:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 17, 2026
@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@rascani

rascani commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

@claude /pr-review

Please review the current revision (6267fb63f782406981806724007bbd7529e4fa69) against this PR's base branch, codex/espressif-fused-quant-activation-fusion. This PR is part of stack #25158. Re-check any previous findings against the updated code and focus on actionable correctness, quantization/numerical, memory-safety, and test-coverage issues.

@claude

claude Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Claude finished @rascani's task in 20m 40s —— View job


I found no correctness, quantization, or memory-safety bugs in the linear + ReLU/HardTanh fusion at 6267fb6, only two minor follow-ups: range-check the new clamp bounds in the C++ kernel, and make the quantizer bias test assert that grouping happened.

Detailed review

PR Review: #25178

Summary

The PR groups single-consumer linear → ReLU/HardTanh in the quantizer. It folds the activation's output qparams and clamp into fused_quant.linear, and passes the new activation_min/activation_max through the quantized_linear_int8 schema, the reference implementation, and the ESP-NN kernel.

I found no correctness, numerical, or memory-safety defects in the quantizer grouping, FoldActivations, or the lowering. The two issues below are minor. There were no earlier review findings on this PR to re-check. All 9 changed files were reviewed. No tests were run in this job, and the vendored ESP-NN source is not in the checkout.

Code Quality

  • backends/espressif/ops/op_quantized_linear.cpp:76-77: the new clamp bounds are never range-checked. activation_min and activation_max are narrowed with static_cast<int32_t> and passed to ESP-NN without a check.

    • Trigger: a malformed or hand-edited program, for example activation_max = 200 or activation_min > activation_max. The lowering itself never emits such bounds, because _qparams raises on an invalid range (lower_to_esp.py:156-159).
    • Effect: the kernel returns wrong int8 output instead of failing with InvalidArgument. It does not read or write out of bounds.
    • Precedent: the ESP-NN delegate runtime already rejects the same values with valid_activation_range (espnn_runtime/src/xts3_graph.cpp:69-71).
    • Fix: add ET_KERNEL_CHECK(ctx, std::numeric_limits<int8_t>::min() <= activation_min && activation_min <= activation_max && activation_max <= std::numeric_limits<int8_t>::max(), InvalidArgument, out). <limits> is already included.
    • Same gap elsewhere: the add and mul kernels from the base commit (op_quantized_add.cpp:69-70, op_quantized_mul.cpp:63-64,76-77) need the same check.

    Fix this →

Testing

  • backends/espressif/test/test_fused_quant_quantizer.py:92-108: the new activation parametrization never checks that grouping happened.

    • Problem: the test still passes if torch.ops.aten.linear.default is removed from the grouping list (esp_fused_quantizer.py:165). So it does not show that per-channel weights and the derived int32 bias survive grouping for the vector, 2-D, and rank-3 shapes.
    • Current coverage: grouping is only asserted in test_linear_activation_quantization_boundary, at shape (2, 17).
    • Fix: add assert (get_qparams_from_node(node, "out") is None) == (activation is not torch.nn.Identity).

    Fix this →

Recommendation

Approve. Neither finding blocks the PR. Both are small hardening and test-strength follow-ups.

Review steps
  • Gather context (PR metadata, CLAUDE.md, review checklist)
  • Fetch the diff against codex/espressif-fused-quant-activation-fusion (1 commit, 9 files, +200/−31)
  • Run independent reviews (3 reviewers, all 9 files covered)
  • Consolidate and deduplicate findings (7 candidates)
  • Fact-check against the code (5 dropped: wrong, already covered elsewhere, or not actionable)
  • Post final review

Not done: no Python or C++ tests were run, because running Python needs approval this job does not have.

@rascani
rascani force-pushed the codex/espressif-fused-quant-linear-activation-fusion branch from 6267fb6 to fe3fa73 Compare November 17, 2026 22:05
@rascani
rascani force-pushed the codex/espressif-fused-quant-linear-activation-fusion branch from fe3fa73 to 07491f0 Compare November 17, 2026 23:40
@rascani
rascani force-pushed the codex/espressif-fused-quant-linear-activation-fusion branch from 07491f0 to b659712 Compare November 18, 2026 00:39
@rascani
rascani force-pushed the codex/espressif-fused-quant-linear-activation-fusion branch from 418b7c5 to 14266b4 Compare November 18, 2026 20:52
@rascani
rascani requested a review from JakeStevens November 18, 2026 21:30
@rascani
rascani marked this pull request as ready for review November 18, 2026 21:30
@rascani
rascani force-pushed the codex/espressif-fused-quant-linear-activation-fusion branch 2 times, most recently from ac600fd to b722d7b Compare November 18, 2026 23:56
@rascani
rascani removed this pull request from stack #25158 November 18, 2026 23:59
@rascani
rascani added this pull request to stack #25261 November 19, 2026 00:01
@rascani
rascani removed this pull request from stack #25261 November 19, 2026 00:23
@rascani
rascani force-pushed the codex/espressif-fused-quant-linear-activation-fusion branch from b722d7b to 56c4efc Compare November 19, 2026 00:23
@rascani
rascani added this pull request to stack #25262 November 19, 2026 00:23
Quantize single-consumer linear followed by ReLU or HardTanh, including
ReLU6, at the pattern boundaries. Fold the final output qparams and clamp
bounds into the existing ESP-NN fully connected call, preserving per-channel
weights and derived int32 bias. Forked and separately quantized activations
retain their original quantization boundaries.

Read the quantizer and activation fold before the linear lowering and
schema/wrapper updates. Shared dialect and S3 cases cover bias/no bias,
forks, saturation, vector and higher-rank inputs, and narrower clamp grids
with asymmetric, positive-only and negative-only HardTanh bounds.

Authored with OpenAI Codex.

This branch was previously deployed

1 inactive deployment
cadence — 93263ca2 Deployed Nov 21, 2026 by rascani via hifi5s-op-test / hifi5s #39864
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant