Skip to content

[Espressif] Bind ESP-NN scratch-size queries by target - #25157

Open
rascani wants to merge 3 commits into
codex/espressif-fused-quant-linearfrom
codex/espressif-scratch-bindings
Open

rascani wants to merge 3 commits into
codex/espressif-fused-quant-linearfrom
codex/espressif-scratch-bindings

Conversation

@rascani

@rascani rascani commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

ESP operator lowering needs the scratch-size calculations for the selected device kernel without building or running ESP-NN kernels on the host. This adds optional bindings for all seven ESP-NN 1.5.0 scratch-size APIs: convolution, depthwise convolution, softmax, mean, hard-swish, batched fully connected, and logistic.

EspTarget and EspKernelMode select S3, P4/S31 PIE, generic optimized, or ANSI queries. The pybind11 functions have typed parameters and named keyword arguments. CMake uses the delegate's shared ESP-NN fetch, with one default release tag and the same ESPNN_LOCAL_PATH override. Local checkouts require no version metadata. The root CMake size-optimized build retains the C++ exceptions and RTTI needed by pybind11, and CI exercises that configuration. The host modules link sizing code only; MSVC-style compiler frontends receive an explicit unsupported-toolchain error.

The bindings check dimension bounds and intermediate arithmetic before entering vendor code, including combinations whose overflow would otherwise return a plausible positive size. Actual sizes still come from the vendor getters. Missing optional modules permit portable fallback; ABI failures and missing transitive dependencies surface as errors. CI requires all four profiles so absent extensions cannot turn the suite green through skips.

Review the selector and binding API first, then build/package integration and tests. Conv2d lowering and runtime scratch use remain in #25154.

Validation: standalone local-source, pinned fetched-source, and root-CMake sizing-only builds passed. All four size-optimized modules passed the 222 sizing/selector/bounds tests, and a combined delegate/scratch build passed three host interpreter tests. Default fetching and a local checkout without a version manifest both configured successfully; the missing-bindings negative control failed as intended. These tests also pass within the revised stack's 978-test Python suite. Scoped lint passed. Additional chip profiles have host sizing coverage only; wheel-install CI remains follow-up coverage.

Authored with OpenAI Codex.

@pytorch-bot

pytorch-bot Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/25157

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 2 Unclassified Failures

As of commit 5d8793f with merge base 351e938 (image):

NEW FAILURES - The following jobs have failed:

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@rascani
rascani added this pull request to stack #25158 November 17, 2026 04:00
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 17, 2026
@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@rascani

rascani commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

@claude review effort=xhigh

@claude

claude Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Claude finished @rascani's task in 3m 41s —— View job


This PR adds optional ESP-NN 1.5.0 scratch-size pybind11 modules (one per kernel profile) and an EspTarget/EspKernelMode selector, and I'd request changes because a broken or missing binding makes the new CI step skip its tests while still passing, and the pip install build path is never tested.

Detailed review

PR Review: #25157

Summary

The PR adds four profile-specific host modules (esp32s3, riscv_pie, generic, ansi), each exposing seven ESP-NN scratch-size queries, plus a Python selector that picks a module from the target and kernel mode.

The main problems:

  • If a binding is missing or fails to load, the new CI step skips its tests and still passes.
  • The documented pip install build path is never exercised in CI.
  • Input validation covers only the convolution queries, and it can't stop overflow in the vendor's own arithmetic.

The ESP-NN sources aren't on this runner and network access was blocked. Every claim below that depends on vendor arithmetic or parameter types is marked as suspected.

API Design

  • backends/espressif/scratch/__init__.py:27-32: except ImportError: return None hides real load failures.
    • A module that was built but won't load (for example an undefined symbol error, which is plausible given C_VISIBILITY_PRESET hidden plus --gc-sections) looks exactly like "not built".
    • The lowering code in [Espressif] Lower Fused Quant Conv2d with planned scratch #25154 then silently gets None, and the user never sees the real error.
    • Fix: catch only ModuleNotFoundError where e.name is the requested module name. test_missing_binding still passes with that change, because a None entry in sys.modules raises ModuleNotFoundError with name set to that module.
    • Fix this →
  • backends/espressif/scratch/bindings.cpp:107-127: validation and errors differ from query to query.
    • Only conv and depthwise check their arguments and raise ValueError.
    • fc_batch_scratch_size rejects out-of-range values only through pybind's uint16_t conversion, so it raises TypeError. test_scratch.py:54 locks that behaviour in as if it were the API.
    • softmax_scratch_size and mean_scratch_size pass values straight to the vendor code, so negative values go through unchecked if those parameters are int32_t (suspected; the vendor header wasn't checked). batches is never tested.
    • The README documents only the conv behaviour.
    • Either validate every query the same way (one error type, documented), or document per query what is accepted.
  • bindings.cpp:112-117: argument order is inconsistent. mean_scratch_size takes (height, width, channels), while conv and softmax take width first. Keyword names reduce the risk, but anyone calling positionally will swap the axes without noticing. Consider making them keyword-only (py::kw_only()), or reorder to match the others.

Infrastructure

  • .ci/scripts/test_espressif.sh:163: the new step passes even when every vendor test is skipped.
    • Every test that needs a binding calls pytest.skip when get_scratch_queries returns None, and pytest exits 0 when tests are skipped.
    • This happens if CMAKE_LIBRARY_OUTPUT_DIRECTORY points somewhere other than the directory pytest imports executorch from, if the module has the wrong suffix, or if it fails to load (see the ImportError item above). In all of these cases only the three selector tests run and CI is green.
    • Fix: make a missing binding a failure in CI, for example with an env var that turns the skip into a failure, or with a direct import of all four modules before running pytest.
  • setup.py:2919-2920, 3334-3346; CMakeLists.txt:898-900: the build path the README documents is never exercised.
    • CI only builds the modules with a standalone cmake -S backends/espressif/scratch. It never builds them through CMAKE_ARGS=-DEXECUTORCH_BUILD_ESP_NN_PYBINDS=ON pip install.
    • So none of these is tested: the root add_subdirectory, the _esp_nn_scratch target, the BuiltExtension src_dir/glob, and the FetchContent path. Regressions in any of them would ship unnoticed.
    • .github/workflows/test-espressif.yml path filters also don't include setup.py, the root CMakeLists.txt or tools/cmake/preset/**.
  • .ci/scripts/test_espressif.sh:157-161: CI writes build output into the package directory. Setting CMAKE_LIBRARY_OUTPUT_DIRECTORY to esp.__path__ writes the .so files into the source tree (editable install) or into site-packages, so the job leaves untracked artifacts behind.
  • backends/espressif/scratch/CMakeLists.txt:180-184: the download has no integrity check.
    • The fetched GitHub tag tarball has no URL_HASH SHA256=..., so "pinned" in the README is really "pinned by tag".
    • ESP_NN_LOCAL_PATH isn't checked to exist or to be version 1.5.0. A wrong path fails late with an unclear "Cannot find source file" error. A different version silently returns different sizes.
    • SOURCE_SUBDIR src/convolution is a non-obvious trick: it stops MakeAvailable from adding ESP-NN's top-level IDF-component CMakeLists.txt. It deserves a one-line comment, or use FetchContent_Populate-style population instead.
  • backends/espressif/scratch/CMakeLists.txt:241-246: the build flags assume GCC/Clang. -ffunction-sections -fdata-sections and -Wl,--gc-sections are added on every platform. Meanwhile setup.py adds a Windows-specific .cp* glob, which implies Windows is supported. Turning the option on with MSVC will fail to link. Either guard the flags (if(NOT MSVC)) or reject the option on Windows.

Code Quality

  • bindings.cpp:22-55: the bounds check doesn't prevent vendor integer overflow (suspected).
    • Each value is limited to 65535 on its own, but 65535 * 65535 already exceeds INT32_MAX.
    • Inputs such as input_dims=(65535, 65535, 1) or filter_dims=(65535, 65535, 16) pass validation. They then likely hit signed-overflow undefined behaviour in the S3/PIE getters, which multiply width × height × channels as int. The result goes back to Python unchecked and may be negative.
    • At minimum, reject a negative return value. Better, bound the products the vendor code computes.

Testing

  • test_scratch.py:104-125: the boundary test asserts only >= 0. It uses degenerate shapes with only one axis at 65535, so it can't catch the overflow case above or a wrong size. Assert exact expected values, and add a case with several large axes at once.
  • test_scratch.py:14-43: the parametrization repeats the same modules. It runs all 13 EspTarget values × 2 modes, but there are only four distinct modules: 13 of the 26 cases load ansi, and 10 load the same generic module. In particular, the ESP32-S31 case loads the same .so as ESP32-P4 (built with only CONFIG_IDF_TARGET_ESP32P4=1, CMakeLists.txt:236). It doesn't confirm that ESP-NN 1.5.0 configures S31 the same way, and that mapping is asserted, not derived from esp_nn.h (suspected). The "161 tests" figure overstates the real coverage.

Recommendation

Request Changes

Blocking: the CI step can't fail when the bindings are missing or broken, and the pip install path it ships is untested. Narrow the ImportError catch and make CI fail when the bindings are absent. The other items are minor or low confidence and need the vendor source to confirm.

@rascani
rascani requested a review from JakeStevens November 18, 2026 00:59
@rascani
rascani marked this pull request as ready for review November 18, 2026 01:00
@rascani
rascani force-pushed the codex/espressif-scratch-bindings branch from 95a06a1 to 653ddfa Compare November 18, 2026 18:17
@rascani
rascani force-pushed the codex/espressif-scratch-bindings branch from 653ddfa to 10bd570 Compare November 18, 2026 20:52

@JakeStevens JakeStevens left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sure:

catch only ModuleNotFoundError where e.name is the requested module name. test_missing_binding still passes with that change, because a None entry in sys.modules raises ModuleNotFoundError with name set to that module.

is fixed

@JakeStevens JakeStevens left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sure:

catch only ModuleNotFoundError where e.name is the requested module name. test_missing_binding still passes with that change, because a None entry in sys.modules raises ModuleNotFoundError with name set to that module.

is fixed

@rascani

rascani commented Nov 21, 2026

Copy link
Copy Markdown
Contributor Author

@claude /pr-review recheck findings

@claude

claude Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

Claude finished @rascani's task in 47m 22s —— View job


PR Review: #25157 (recheck)

JakeStevens' import-fallback condition is now met, but the code cited by the 919c812 review hasn't changed, so its eight findings are still open, five new ones were found, and the verdict stays Request Changes.

Detailed review

Summary

This PR adds optional pybind11 bindings for ESP-NN 1.5.0's scratch-size getters, with one module per kernel profile. It also adds an EspTarget/EspKernelMode selector and host tests that run in the optimized emulator jobs.

Recheck:

  • JakeStevens' condition, catching only ModuleNotFoundError with error.name == module_name, is met at backends/espressif/scratch/__init__.py:28-33. The first review's fixes still hold.
  • Every line the 919c812 review cited is unchanged, so findings Add support for quantized LeakyReLU #1–Executorch OSS CI tracking issue (2023) #8 and the mean-order note are still open. They are tagged still open with their earlier number; five findings are tagged new.
  • I withdraw the earlier note about writing into the package directory.
  • ESP-NN sources and network access were unavailable in this run, so claims about vendor internals are marked suspected.

Code Quality

  • Question, still open (Rename _pt2e to pt2e #3): which buffer does fc_batch_scratch_size size? S3 and PIE report 704 and 800 bytes for (8, 16, 32) (test_scratch.py:33-35). However, op_quantized_linear.cpp:62-76 calls esp_nn_fully_connected_per_ch_s8_batch without a scratch buffer, and the runtime sets only the conv, depthwise and mean buffers (int8_interpreter.cpp:1972-1974, 1983-1985). Either batched FC on S3/P4 needs a buffer that nothing provides (a suspected device memory-safety issue), or this query sizes something no caller can use. Please answer this before the API ships.

  • Minor, still open (Add support for quantized LeakyReLU #1): softmax and mean have no overflow guard for the S3/PIE getters, and no getter result is checked.

    • Softmax checks only width * 4 (bindings.cpp:141-149), and mean checks only that its inputs are positive (151-158).
    • S3/PIE softmax returns 2064 for (32, 8), while generic returns 32 * 4 = 128 (test_scratch.py:33-37). Their formula must be different, and height has no bound.
    • The S3 mean getter is compiled in (scratch/CMakeLists.txt:72) with no upper bound on its inputs.
    • No return value is checked for < 0 (lines 106, 137, 148, 157, 173), so a wrapped size would reach Python as a valid one. This is suspected; it needs shapes far beyond MCU sizes.

    Fix this →

  • Minor, new: the convolution/FC bound counts max(out_c, in_c * mult) output channels, so the FC bound grows with row_len².

    • The bound is at bindings.cpp:58-59 and 72-73, and FC calls it at 165-172.
    • For FC, filter_bound = (row_len + 15) * row_len, so every row_len above 32736 raises ValueError, whatever out_ch is.
    • Example: fc_batch_scratch_size(1, 32768, 10) computes a bound of 2,150,613,248. That is a 32×32×32 feature map flattened into 10 classes, with 320 KiB of weights, and op_quantized_linear.cpp:37-42 accepts the shape.
    • Convolution with in_c > out_c is over-counted the same way.
    • test_fc_arithmetic_bounds (test_scratch.py:185-187) trips on row_len alone.
    • Confirmed by hand arithmetic.

    Fix this →

  • Minor, still open (Re-sync with internal repository #2): targets outside the mapping silently get the generic profile. scratch/__init__.py:22-26 uses .get(target, "generic"). A future EspTarget member with PIE kernels would get generic sizes instead of an error: 0 bytes where PIE needs 1808 for the conv case in test_scratch.py:35. Fix this →

  • Minor, new: a second ESP-NN pin, with a differently spelled override. scratch/CMakeLists.txt:22-52 pins the v1.5.0 tarball and reads ESP_NN_LOCAL_PATH. espnn_runtime/host/espnn_host.cmake:15-37 pins ESPNN_GIT_TAG and reads ESPNN_LOCAL_PATH. The README documents both spellings (README.md:52, :501). Every ESP-NN bump now has to update both pins, the version regex and the component manifests together. Fix this →

Infrastructure

  • Major, still open (Add unlifting pass under private config #4): the new path filters run the workflow for build paths that no job exercises.

    • .github/workflows/test-espressif.yml:29-32 adds setup.py, CMakeLists.txt and tools/cmake/preset/default.cmake.
    • Every job installs with --minimal (lines 85, 107, 129, 155), which forces -DEXECUTORCH_BUILD_ESP_NN_PYBINDS=OFF (setup.py:176).
    • The CI scratch step configures backends/espressif/scratch standalone (test_espressif.sh:185).
    • So three paths are never built: the root hook (CMakeLists.txt:898-900), the setup.py target and extensions (2917-2918, 3332-3344), and the documented pip install (README.md:48). tools/cmake/preset/pybind.cmake:9 enables the CMSIS-NN bindings but not these.
    • Every PR that touches those three common files now pays for 8 jobs on mt-l-x86iavx512-8-64. Meanwhile third-party/pybind11, which the bindings compile against, is not a trigger.

    Fix this →

  • Minor, new: scratch sizing runs inside the emulator jobs.

    • .ci/scripts/test_espressif.sh:183-191 builds all four host modules, which don't depend on the target, and runs test_scratch.py in each of the three optimized jobs.
    • Under set -euo pipefail (line 8), a host-side failure (logged returns 1, line 148) exits before the operator-library device tests (lines 193-201), so that run loses their results.
    • Because ESP_NN_LOCAL_PATH points at the managed component, CI never runs the default pinned download and SHA-256 check (scratch/CMakeLists.txt:41-52).

    Fix this →

  • Minor, new: EXECUTORCH_OPTIMIZE_SIZE breaks the bindings' build. The root CMakeLists.txt:341-347 appends -fno-exceptions -fno-rtti to CMAKE_CXX_FLAGS_RELEASE before add_subdirectory(backends/espressif/scratch) at line 899. bindings.cpp throws pybind11::value_error, so a Release configure with both options fails to compile. Other pybind targets turn exceptions and RTTI back on (_pybind_compile_options at CMakeLists.txt:1468-1476, tools/cmake/Codegen.cmake:332). Confirmed by reading the flags, not by building. Fix this →

Testing

  • Major, new: host sizes are never compared with the device getters.

    • The expected tuples in test_scratch.py:30-37 pin the host build's own output. Nothing shows they match the firmware's results, even though the emulator jobs build both from the same managed component.
    • The vendor arithmetic runs on LP64 x86-64, not the 32-bit target.
    • S31 is compiled as CONFIG_IDF_TARGET_ESP32P4 (scratch/CMakeLists.txt:101-102). Whether ESP-NN treats S31 differently is suspected.
    • The generic profile has no device job.
    • If the host and device sizes differ, any consumer that plans scratch from these values under-allocates on the device.
    • README.md:91-92 says device tests cover ESP32-S3, but the operator-library device tests also run on P4 and S31: test/tester.py:161 reads the target that test_espressif.sh:22 exports.

    Fix this →

  • Minor, still open ([build] Add BUCK.oss #5): the aggregate bound, argument orientation and zero-result profiles are untested.

    • All four test_convolution_arithmetic_bounds cases (test_scratch.py:139-142) fail inside input_bound's checked_product. That includes filter_product, because extent contains the filter. So filter_bound, output_bound, window_bound and the sum at bindings.cpp:78-84 could be deleted and every test would still pass.
    • Input (8191, 8191, 16), filter (1, 1, 16), output (8191, 8191, 16), stride (1, 1), padding (0, 0) is rejected only by the sum (3,220,981,872 > INT32_MAX).
    • Stride and padding are symmetric in every case.
    • Mean is tested only at height == width, where every profile returns 0.
    • PIE depthwise returns 0 in both shape tests, so it can't be told apart from generic.
    • FC batches is never tested at 65535 or at the int32 limit.

    Fix this →

API Design

  • Major, still open (Re-sync with internal repository #6): the getter inputs differ from what the runtime passes.

    • The bindings set data_dims_t.extra = 0 (bindings.cpp:99-101, 129-131); the runtime uses 1 (int8_interpreter.cpp:743-745).
    • For depthwise, the runtime passes the output channels as the filter channels and derives ch_mult from them (int8_interpreter.cpp:824-828). The tests pass input channels, for example filter (3, 2, 16) with 32 output channels (test_scratch.py:38-47, 124-131). Nothing checks out_c == in_c * channel_multiplier.
    • Dilated convolutions are sized per phase tile with stride 1 and no padding (int8_interpreter.cpp:758-805), while the README says only "unit dilation" (README.md:75-76).

    If a vendor getter reads these fields (suspected), an AOT planner gets different sizes from the device's require_scratch, most likely smaller ones. The AOT sizing and the runtime need one shared definition of these inputs. Fix this →

  • Minor, still open (change internal code to only import from _quantize_pt2e #7): nothing ties the host profile or ESP-NN version to the firmware. The caller picks EspTarget/EspKernelMode, and nothing records them or checks them against the firmware's IDF_TARGET/CONFIG_NN_*. For example, sizing with ANSI for optimized firmware returns 0 everywhere. examples/espressif/idf_component_template/README.md:96 also allows a local ESP-NN "1.5.0 or newer", while the bindings require exactly 1.5.0. Fix this →

  • Minor, still open (Executorch OSS CI tracking issue (2023) #8): out-of-range ints raise TypeError, not the documented ValueError. The parameters are int32_t or std::array<int32_t, N>. pybind11 rejects calls such as fc_batch_scratch_size(2**31, 16, 32) or input_dims=(2**31, 1, 1) during overload resolution, with TypeError: incompatible function arguments. README.md:83 says invalid bounds raise ValueError. This follows pybind11's integer caster behavior; I didn't run it here. Fix this →

  • Minor, still open: mean_scratch_size takes (height, width, channels) (bindings.cpp:151, 207-212). The module docstring (bindings.cpp:182-184) and every other query put width first. A positional call in docstring order silently transposes the query. Fix this →

Recommendation

Request Changes

Blocking:

The host-to-device size comparison can land with #25154, but it should exist before anything plans scratch from these values. Everything else is minor. JakeStevens' condition is met.

  • Gather context
  • Fetch diff
  • Run three independent reviewers
  • Consolidate findings
  • Fact-check and re-check each earlier finding
  • Post review

Expose all seven ESP-NN 1.5.0 scratch-size APIs through typed pybind11
functions with named arguments. EspTarget and EspKernelMode select S3,
RISC-V PIE, generic optimized, or ANSI queries from the vendor sources.

Keep the bindings optional in main CMake and Python packaging, with a
standalone sizing-only build and a local source override. Verify query
results and dimension bounds in the optimized ESP CI job. Operator
lowering and device scratch allocation are introduced separately.

Validated standalone and main-CMake builds, 33 sizing/selector tests,
and lintrunner.

Authored with OpenAI Codex.
Reject convolution and depthwise convolution dimensions, strides, and
channel multipliers outside [1, 65535], and padding outside [0, 65535],
before calling ESP-NN scratch-size queries. Vendor dispatch narrows these
values to uint16; a zero or wrapped filter width can otherwise terminate
Python with a divide-by-zero instead of raising an exception.

Document the bounds and exercise invalid values and valid boundaries
across the S3, RISC-V PIE, generic, and ANSI bindings. The sizing-only
binding build, 161 scratch tests, and lintrunner passed.

Authored with OpenAI Codex.
Enable the C++ exceptions and RTTI required by pybind11 when the root build disables them for the runtime. Exercise this configuration in CI by building scratch bindings through root CMake with size optimization enabled.

Use the shared ESP-NN source fetch and ESPNN_LOCAL_PATH override, removing the separate download pin and local version checks.

Authored with OpenAI Codex.

This branch was previously deployed

1 inactive deployment
cadence — 5d8793f5 Deployed Nov 21, 2026 by rascani via hifi-op-test / hifi4 #39854
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants