Repository navigation
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/25149
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 3c3ca5f with merge base 351e938 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
66d6a20 to
791bc08
Compare
|
@claude review |
|
Claude finished @rascani's task in 2m 34s —— View job This PR moves the ESP-NN pin to 1.5.0 and removes the square-output guards on AvgPool2d. I found no blockers, only two minor test gaps and one minor doc gap for projects that bring their own ESP-NN. Detailed reviewPR Review: #25149SummaryThe PR pins all ESP-NN dependencies to 1.5.0 and removes the rectangular-output guards in AvgPool2d lowering and in the kernel wrapper, because 1.5.0 fixes the S3 row-stride bug. The issues are minor: a project that brings its own ESP-NN gets no minimum version, and there are two test coverage gaps. Infrastructure
Testing
RecommendationApprove Nothing blocks merging. The README line for vendored ESP-NN is the most useful follow-up. |
791bc08 to
85163e8
Compare
791d8f1 to
29ecdcd
Compare
29ecdcd to
aeaf8ea
Compare
|
Hi @rascani, ESP-NN maintainer here. Thanks for keeping the backend current with ESP-NN. With ESP-NN 1.5.0, two workarounds in the delegate's
Both workarounds also sent these shapes to the ANSI kernels on every other chip. With this PR's 1.5.0 pin they can go. The patch below removes them and adds two 1x1 cases over 256 x 257 = 65792 positions, one per ESP32-S3 pointwise path (input channels a multiple of 8, and not). Patch on top of this PRdiff --git a/backends/espressif/espnn_runtime/src/int8_interpreter.cpp b/backends/espressif/espnn_runtime/src/int8_interpreter.cpp
index 0481746..227da1c 100644
--- a/backends/espressif/espnn_runtime/src/int8_interpreter.cpp
+++ b/backends/espressif/espnn_runtime/src/int8_interpreter.cpp
@@ -209,15 +209,6 @@ int8_t quantize_value(float value, float inverse_scale, int32_t zero_point) {
static_cast<int32_t>(std::nearbyintf(scaled)) + zero_point);
}
-// esp-nn's ESP32-S3 pointwise convolution passes the number of output
-// positions as a uint16_t width and leaves the rest unwritten.
-bool pointwise_beyond_uint16(
- const data_dims_t& filter,
- const data_dims_t& output) {
- return filter.width == 1 && filter.height == 1 &&
- static_cast<int64_t>(output.width) * output.height > UINT16_MAX;
-}
-
} // namespace
// Fields that no operation uses together share storage: the input list
@@ -1226,7 +1217,7 @@ bool Int8Interpreter::run_operation(uint32_t index) {
run_generic_conv(s, input, output);
return true;
}
- if (reference || pointwise_beyond_uint16(s.filter_dims, s.output_dims)) {
+ if (reference) {
esp_nn_conv_s8_ansi(
&s.input_dims,
input,
@@ -1380,9 +1371,7 @@ bool Int8Interpreter::run_operation(uint32_t index) {
sw = s.pool[3];
const uint16_t ph = s.pool[4], pw = s.pool[5];
if (s.opcode == OpCode::MAX_POOL2D) {
- // The ESP32-S3 kernel takes a multiple of four channels and writes
- // nothing otherwise.
- if (reference || channels % 4 != 0) {
+ if (reference) {
esp_nn_max_pool_s8_ansi(
input,
in_w,
@@ -1676,7 +1665,7 @@ void Int8Interpreter::run_dilated_by_phases(
data_dims_t tile_dims{tile_w, tile_h, in_c, 1};
data_dims_t result_dims{cols, rows, out_c, 1};
if (s.opcode == OpCode::CONV2D) {
- if (reference || pointwise_beyond_uint16(s.filter_dims, result_dims)) {
+ if (reference) {
esp_nn_conv_s8_ansi(
&tile_dims,
tile,
diff --git a/backends/espressif/test/test_int8_ops.py b/backends/espressif/test/test_int8_ops.py
index 7e5c1a9..9935de1 100644
--- a/backends/espressif/test/test_int8_ops.py
+++ b/backends/espressif/test/test_int8_ops.py
@@ -248,6 +248,24 @@ class ConvAvgPool(nn.Module):
return F.avg_pool2d(self.conv(x), 2)
+class PointwiseConv(nn.Module):
+ """A 1x1 convolution over 256 x 257 = 65792 positions, past uint16_t.
+
+ Nearest upsampling builds the large map inside the delegate, after a
+ small convolution quantizes it, and the large convolution has one output
+ channel, so the bundled inputs and outputs stay small.
+ """
+
+ def __init__(self, channels):
+ super().__init__()
+ self.stem = nn.Conv2d(channels, channels, 1)
+ self.conv = nn.Conv2d(channels, 1, 1)
+
+ def forward(self, x):
+ y = F.interpolate(self.stem(x), size=(256, 257), mode="nearest")
+ return self.conv(y)
+
+
class GroupedConvs(nn.Module):
"""Grouped (not depthwise) convolutions: plain, strided and dilated."""
@@ -284,6 +302,10 @@ CASES = [
("two-inputs-two-outputs", TwoInputTwoOutput(), [(1, 3, 12, 12), (1, 6)], 2),
("transpose-select-stack", TransposeSelect(), [(1, 3, 4, 5)], 2),
("max-pool-six-channels", OddChannelPool(), [(1, 3, 8, 8)], 2),
+ # One case per ESP32-S3 pointwise path: input channels a multiple of 8,
+ # and not.
+ ("pointwise-65792-positions-c8", PointwiseConv(8), [(1, 8, 8, 257)], 2),
+ ("pointwise-65792-positions-c4", PointwiseConv(4), [(1, 4, 8, 257)], 2),
("avgpool-wide-c4", ConvAvgPool(4), [(1, 4, 5, 7)], 2),
("avgpool-tall-c4", ConvAvgPool(4), [(1, 4, 7, 5)], 2),
("avgpool-wide-c8", ConvAvgPool(8), [(1, 8, 5, 7)], 2),We ran these three cases on top of this PR with the patch, through the CI script's delegate flow in esp-emulator 1.1.0, optimized kernels:
All three are fully delegated and match the bundled reference at the exact tolerance. |
Pin the delegate, operator-library, and packaged ESP-IDF components to ESP-NN 1.5.0 for the S3 AvgPool row-stride fix, faster multiply, and convolution/pooling alignment fixes. Remove the temporary rectangular AvgPool checks from lowering and the kernel wrapper. Exercise wide and tall outputs through the shared operator cases and four delegate cases. All 76 operator-library cases and 20 delegate programs in each of optimized and ANSI modes pass on the S3 emulator. Generated component manifest validation and scoped lintrunner pass. Compared with 1.4.2, matching test firmware grows by 2896 bytes for the operator library and 4512 bytes for the optimized delegate; ANSI shrinks by 224 bytes. The new S3 multiply frames require 216 additional stack bytes. Authored with OpenAI Codex.
Let ESP-NN handle large pointwise convolution maps and max pooling with channel counts not divisible by four. Version 1.5.0 chunks large S3 pointwise calls and handles pooling alignment and channel tails, so the delegate no longer needs to force those cases to ANSI. Keep the explicit reference path in ordinary and phase-based convolution and max pooling. Add pointwise regressions with 65,792 output positions and four or eight input channels. Those cases and the existing six-channel max-pool case pass the host reference tests and the ESP32-S3 emulator in both optimized and ANSI configurations. Scoped lintrunner and git diff --check pass. Authored with OpenAI Codex.
|
@claude /pr-review recheck findings |
|
Claude finished @rascani's task in 30m 3s —— View job Re-checked the earlier reviews' findings at Detailed reviewPR Review: #25149SummaryThe PR pins every ESP-NN dependency to 1.5.0 and removes the AvgPool2d square-output guard. The new commit Status of the earlier findings:
There are two new findings: the overview doc is stale, and the pointwise cases can't detect a misplaced chunk. I didn't run any tests. ESP-NN's sources aren't in the checkout, so what I say below about vendor behaviour comes from the comments this PR removes and from the maintainer's note. Code Quality
Infrastructure
Testing
Backward Compatibility
RecommendationApprove Nothing blocks the default managed build, which can only resolve 1.5.0. Update The configure-time version check is still the most useful follow-up. Without it, a stale local ESP-NN turns the removed guards into silently wrong results, and into memory corruption for tall AvgPool outputs. |
Let ESP-IDF resolve compatible releases through the component manifests. Move the delegate's host source fetch into a shared CMake file with one default pin and ESPNN_LOCAL_PATH override, without parsing vendor version metadata. Check that the delegate's vendor regression fixtures still lower to ESP-NN, so portable fallback cannot hide a regression. Authored with OpenAI Codex.
Update the default ESP-NN dependency to 1.5.0, including the optimized S3 AvgPool row-stride fix, faster multiply kernels, and convolution/pooling alignment fixes. Rectangular AvgPool outputs now lower to ESP-NN, removing the temporary shape checks from lowering and the kernel wrapper. ESP-IDF manifests accept compatible releases through
^1.5.0ranges. A shared host source fetch has one default release tag and anESPNN_LOCAL_PATHoverride, without custom version checks. The new vendor scratch APIs are left for later operator work.Use the vendor dispatcher for large pointwise convolutions and max pooling with channel counts not divisible by four. ESP-NN 1.5.0 chunks the S3 pointwise calls and handles pooling alignment and channel tails, so the delegate no longer forces those cases to ANSI. Explicit reference execution remains available for ordinary and phase-based convolution and max pooling.
The wide/tall operator cases require acceleration in both channel paths, with rectangular outputs also exercised through padding, batches, chaining, and float consumers. Four Conv → AvgPool delegate cases add coverage to the existing test/export suite. Two pointwise cases cover 65,792 output positions with four or eight input channels, alongside the existing six-channel max-pool regression. Seven public-export assertions verify that these vendor regression fixtures still delegate to the expected ESP-NN operators, so portable fallback cannot mask a regression.
The version upgrade was qualified with the stock registry package on the ESP32-S3 emulator: all 76 operator-library cases passed, and all 20 existing delegate programs passed in both optimized and ANSI modes. The dispatcher cleanup additionally passes the three targeted host reference tests and all three programs in both emulator configurations. The 27 delegate export/reference tests pass with the revised source setup and delegation assertions. Component generation, manifest validation, and scoped lintrunner passed. Physical hardware and dual-core execution were not tested.
The version-only size comparison against matching 1.4.2 test firmware with identical model bundles measured app flash changes of +2896 bytes for the operator library, +4512 for the optimized delegate, and -224 for ANSI. Internal RAM changed by 0, +144, and -16 bytes respectively; the optimized increase was RAM-resident code. These measurements precede the dispatcher cleanup. Separately, both S3 multiply assembly stack frames grow from 120 to 336 bytes.
Authored with OpenAI Codex.