Skip to content

Commit bf9bef3

Browse files
authored
Look up flatc.exe in executorch.data.bin on Windows (#23992)
This PR was authored with AI assistance (Claude Code); I reviewed every changed line and take responsibility for it. ### Summary Fixes #23989 On Windows the wheel packages flatc through `setup.py`'s `get_executable_name()`, so the binary is `executorch/data/bin/flatc.exe`. `_get_flatc_path()` in `exir/_serialize/_flatbuffer.py` asked `executorch.data.bin` for `flatc`, `is_file()` was `False`, and the lookup fell back to a bare `flatc` on `PATH`. That resolves only when the environment's `Scripts` directory is on `PATH` (pip puts the `flatc` console-script shim there). Called by the interpreter's full path, `to_executorch()` after `XnnpackPartitioner` raised `FileNotFoundError: [WinError 2]`. The change asks for `flatc.exe` on `win32` and leaves the other platforms and the first candidate (`flatbuffers-flatc`, used by the Buck/PAR build) as they were. This completes #19899, which added the `executorch.data.bin` candidate with the Linux/macOS spelling only. All eight modules that run flatc import the same helpers from this file, so no other call site needs a change; the audit is in #23989. ### Test plan Added `TestGetFlatcPath` to `exir/_serialize/test/test_flatbuffer.py`. It fakes an `executorch.data.bin` directory containing only the wheel's filename and checks that `_get_flatc_path()` returns that file: `flatc.exe` with `sys.platform` set to `win32`, `flatc` for `linux` and `darwin`. It patches the module's cache globals so it does not depend on test order. ``` python -m pytest exir/_serialize/test/test_flatbuffer.py -k GetFlatcPath python -m pytest backends/xnnpack/test/serialization ``` Both run on the Windows job (`pytest-windows.ini` lists `exir/_serialize/test` and `backends/xnnpack/test/serialization`). Measured on Windows 11 x64, Python 3.12.10, a fresh venv with the PyPI `executorch==1.5.1` win_amd64 wheel and `torch==2.14.0+cpu`, `FLATC_EXECUTABLE` unset, no flatc on `PATH`, interpreter called by full path. I did not build the wheel from source: for "after" I replaced `exir/_serialize/_flatbuffer.py` inside that install with the file from this branch. The `_get_flatc_path()` block in the 1.5.1 wheel is identical to `main`, so the "before" is the same code. ``` before after TestGetFlatcPath (new) 1 failed, 1 passed 2 passed test_serialize_xnnpack_binary (existing) FileNotFoundError passed [WinError 2] exir/_serialize/test + xnnpack serialization, failing test ids 6 4 minimal Linear, to_edge_transform_and_lower + XnnpackPartitioner + to_executorch WinError 2 OK, .pte runs and matches eager ``` The failing set after the change is a strict subset of the set before it: the two that went green are the new test and `test_serialize_xnnpack_binary`, and nothing new fails. The 4 that remain (`test_named_data_store` x2, `test_program` x1, `test_ptn` x1, the last with several failing subtests) fail identically with and without this change. The first three are assertions about named-data alignment and segment handling whose fixes (#23517, #23340) landed on `main` after v1.5.1, so I believe they are `main`'s tests running against the older 1.5.1 wheel code; I did not run them against a build from `main`, and I did not trace the `test_ptn` one. Not covered: I could not run the new test on Linux or macOS (only its `linux`/`darwin` cases with `sys.platform` patched on Windows), and nothing was run under Buck. `ufmt` 2.8.0 / `usort` 1.0.8 / `black` 24.4.2 and `flake8` 6.1.0 at the versions in `requirements-lintrunner.txt` are clean on both files; I did not run the full `lintrunner` (no `mypy`/`pyre`). cc @JacobSzwejbka @angelayi @nil-is-all
1 parent e57ee9c commit bf9bef3

2 files changed

Lines changed: 37 additions & 1 deletion

File tree

‎exir/_serialize/_flatbuffer.py‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
import re
1717
import stat
1818
import subprocess
19+
import sys
1920
import tempfile
2021
import threading
2122

@@ -272,7 +273,12 @@ def _get_flatc_path() -> str:
272273

273274
for package, resource_name in (
274275
(__package__, _FLATC_RESOURCE_NAME),
275-
("executorch.data.bin", "flatc"),
276+
# The wheel installs flatc under the name setup.py's
277+
# get_executable_name() gives it, which has a .exe suffix on Windows.
278+
(
279+
"executorch.data.bin",
280+
"flatc.exe" if sys.platform == "win32" else "flatc",
281+
),
276282
):
277283
try:
278284
flatc_resource = importlib.resources.files(package).joinpath(

‎exir/_serialize/test/test_flatbuffer.py‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
import os
1111
import tempfile
1212
import unittest
13+
from pathlib import Path
1314
from typing import Dict, Optional, Sequence
1415
from unittest.mock import patch
1516

@@ -309,3 +310,32 @@ def test_string_values_are_not_rewritten(self) -> None:
309310
result = json.loads(_replace_non_finite_in_json(content))
310311
self.assertEqual(result["name"], "a:Infinity,b")
311312
self.assertEqual(result["v"], "inf")
313+
314+
315+
class TestGetFlatcPath(unittest.TestCase):
316+
def assert_finds_packaged_flatc(self, platform: str, filename: str) -> None:
317+
"""Checks that flatc is found in executorch.data.bin under its wheel name."""
318+
with tempfile.TemporaryDirectory() as tmp:
319+
bin_dir = Path(tmp, "bin")
320+
bin_dir.mkdir()
321+
(bin_dir / filename).touch()
322+
packages = {
323+
_flatbuffer.__package__: Path(tmp),
324+
"executorch.data.bin": bin_dir,
325+
}
326+
with patch.object(
327+
_flatbuffer.importlib.resources,
328+
"files",
329+
side_effect=packages.__getitem__,
330+
), patch("sys.platform", platform), patch.multiple(
331+
_flatbuffer, _flatc_cached_path=None, _flatc_exit_stack=None
332+
):
333+
self.assertEqual(_flatbuffer._get_flatc_path(), str(bin_dir / filename))
334+
335+
def test_windows_wheel_flatc_exe(self) -> None:
336+
self.assert_finds_packaged_flatc("win32", "flatc.exe")
337+
338+
def test_non_windows_wheel_flatc(self) -> None:
339+
for platform in ("linux", "darwin"):
340+
with self.subTest(platform=platform):
341+
self.assert_finds_packaged_flatc(platform, "flatc")

0 commit comments

Comments
 (0)