Repository navigation
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/25265
Note: Links to docs will display an error until the docs builds have been completed. ❌ 2 New Failures, 3 Cancelled Jobs, 3 PendingAs of commit dbc6f54 with merge base 351e938 ( NEW FAILURES - The following jobs have failed:
CANCELLED JOBS - The following jobs were cancelled. Please retry:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
rascani
force-pushed
the
codex/espressif-mlperf-tiny-coverage
branch
from
November 19, 2026 16:58
43c96cf to
fe97ef3
Compare
rascani
force-pushed
the
codex/espressif-mlperf-tiny-coverage
branch
from
November 20, 2026 18:13
fe97ef3 to
31891a0
Compare
rascani
force-pushed
the
codex/espressif-mlperf-tiny-coverage
branch
from
November 20, 2026 23:00
aef7aad to
5c1649e
Compare
rascani
force-pushed
the
codex/espressif-mlperf-tiny-coverage
branch
from
November 21, 2026 00:02
5c1649e to
64bd749
Compare
Exercise deep autoencoder, DS-CNN, ResNet-8 and MobileNetV1 0.25 through EspTester using deterministic weights, inputs and calibration. Assert accelerated and portable operator inventories, compare lowering with the Fused Quant reference, check serialization, and print Q/DQ and layout counts. Run the same cases on the ESP32-S3 emulator in the optimized CI job. Include the portable padding, mean and softmax kernels needed by the models. Compare classifier probabilities with an absolute tolerance of 1e-3 instead of using intermediate activation scales. Retain the autoencoder quantization tolerance. Authored with OpenAI Codex.
Keep aten::clone.out available in the optimized ESP32-S3 test runtime. The existing Conv2d float_users case returns a constant bias, which requires the clone inserted by constant folding when the program is serialized. Removing the registration in the model coverage change prevented that method from loading. The manual list remains until the deferred PTE tooling can derive the union of required operators from the test programs. Shell syntax and scoped lintrunner pass. Authored with OpenAI Codex.
rascani
force-pushed
the
codex/espressif-mlperf-tiny-coverage
branch
from
November 21, 2026 02:05
64bd749 to
dbc6f54
Compare
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The ESP operator-library tests now cover four complete MLPerf Tiny models: deep autoencoder, DS-CNN, ResNet-8 and MobileNetV1 0.25. The tests reuse the existing model definitions, including softmax outputs, with deterministic synthetic weights, inference inputs and calibration samples.
Each case checks the accelerated and portable operator inventory, compares lowering against the Fused Quant reference, serializes without delegates or residual FQ operators, and reports Q/DQ and layout nodes. The optimized ESP32-S3 CI job runs the same cases through the existing emulator runner and includes the required portable padding, mean and softmax kernels. It retains the clone registration needed by existing tests that return constant tensors.
The remaining portable compute is padding, pooling or mean, and softmax. Layout counts include views that serialization removes from runtime calls. The tests measure compiler/runtime correctness and coverage; synthetic fixtures do not measure trained-model accuracy.
Validation: all 554 operator, pass-manager, and model host tests pass, together with lintrunner and shell syntax checks. The Conv2d float_users implementation case passes on the ESP32-S3 emulator with the complete CI operator selection, confirming the restored clone registration. Local validation uses PyTorch 2.13.0+cpu and TorchAO 0.19.0.dev20261118+cpu; validation with the repository's PyTorch 2.14 pin remains for CI.
Classifier comparisons use an absolute tolerance of 1e-3 in probability space, which rejects injected 32-step classifier errors while accepting normal quantized rounding. The autoencoder retains its quantization tolerance. Earlier runs with a smaller local operator selection passed all four model implementation tests with this bound.
Model execution with the complete CI runtime is currently blocked by the example's 2 MiB app partition: the deep-autoencoder firmware is 2,222,048 bytes. Defer a partition change here while #25287 introduces larger 2944 KiB OTA app slots, then re-run the model implementation tests with that configuration.
Authored with OpenAI Codex.