Skip to content

Commit 0fadbfa

Browse files
authored
recipe: thrift 0.25.0 (#126)
Adds Apache Thrift's Python runtime with its optional C++ codec, `thrift.protocol.fastbinary`, built for Android and iOS. Requested in [flet#6909](flet-dev/flet#6909). - [Recipe](recipes/thrift) - [Docs](recipes/thrift/README.md) - [Example](recipes/thrift/examples/telemetry-rpc) ## Why a recipe Upstream publishes desktop wheels only (cibuildwheel, since 0.24.0) and no pure-Python wheel, so `flet build` has nothing to install on mobile. The `source_packages` workaround suggested in the discussion builds the sdist for the build host instead. pip's isolated build drops serious_python's platform-faking `sitecustomize`, and pip never tag-checks a wheel it built itself. Verified on flet 1.0.3: - The APK carries a macOS Mach-O as `lib/<abi>/libthrift-protocol-fastbinary.so` for all three ABIs. - The iOS app ships a `.fwork` pointing at a framework that was never built. - The app runs on thrift's pure-Python fallback. The `*Accelerated` protocols retry the broken load on every construction and ran 12–23x slower than the plain classes. ## Changes - `CIBUILDWHEEL=1` makes `setup.py` fail when the extension doesn't compile, instead of building a pure-Python wheel. - `mobile.patch` drops `setup.cfg`'s `[install] optimize = 1`. Its byte-compile step imports `distutils` in the cross interpreter and fails on 3.12+. - Android depends on `flet-libcpp-shared` for `libc++_shared.so`. - Skills: - `new-mobile-recipe`: an optional extension is not "pure Python" for `source_packages`; new shape row for forcing optional extensions. - `forge-error-catalogue`: the `source_packages` cause of the iOS lipo error, and the `setup.cfg` `[install] optimize` distutils failure. - `local-recipe-testing`: concurrent iOS builds share `dist_ios`; give each its own `PUB_CACHE`. ## Validation - **CI 6/6** with `mobile_test_pythons=ALL` ([run](https://github.com/ndonkoHenri/mobile-forge/actions/runs/37147858714)): 11 passed, EXIT 0, on cp312, cp313 and cp314 for Android and iOS. - **Tests:** - The codec loads, and the Accelerated protocols bind it. - Output is byte-identical to pure Python, and round trips succeed. - Loopback TCP (framed, buffered, bare) and HTTP (buffered, bare) calls count native encodes and decodes. - All 11 fail against a pure-Python build, which is the negative control. - **Wheels:** Android `DT_NEEDED` lists `libc++_shared.so` with 16 KB `LOAD` alignment; iOS slices are `MH_DYLIB`. - **Example** built and run with flet 1.0.3 on an Android 14 arm64 emulator and an iOS 18.6 simulator: native codec active, RPC and declared exception working, 9–25x native gain for the example payload. - **TLS probe** on both platforms: `TSSLSocket` with the default context verifies against Flet's certifi bundle, and self-signed or expired hosts are rejected. - **Two adversarial review rounds** on the README, example and tests. ## Consumer notes Gives Flet apps a native-speed Thrift client over TCP or HTTP; the generated code is produced on desktop. Install, transport setup, TLS and error handling are in the [recipe README](recipes/thrift/README.md), with a runnable [`telemetry-rpc` example](recipes/thrift/examples/telemetry-rpc).
1 parent 90778e4 commit 0fadbfa

17 files changed

Lines changed: 1614 additions & 1 deletion

File tree

‎.claude/skills/forge-error-catalogue/references/failure-catalogue.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -247,6 +247,15 @@ The reported arch must match the tag — `*-ios_13_0_x86_64_iphonesimulator.whl`
247247

248248
**See also:** `references/recipe-patterns.md` § Pattern F (CMake/Meson) for the full scikit-build-core cross-compile template.
249249

250+
**Second cause, no recipe involved: the app listed the package in `[tool.flet] source_packages`.**
251+
pip then builds the sdist for the build host (its isolated build drops serious_python's
252+
platform-faking `sitecustomize`), so every iOS slice gets the same macOS arm64 `.so` and lipo
253+
refuses the pair. Tells: `build/site-packages/<slice>/<pkg>-*.dist-info/WHEEL` says
254+
`Tag: cp3XX-cp3XX-macosx_*`, and `otool -l` on the `.so` shows `LC_BUILD_VERSION platform
255+
MACOS`. The Android build carries the same Mach-O as `lib/<abi>/lib<pkg>-<mod>.so`. If the
256+
extension is optional the app still runs, on the package's pure-Python fallback. The fix is a
257+
recipe; see the `new-mobile-recipe` skill, "When NOT to use" (thrift).
258+
250259
---
251260

252261
### iOS `flet build ios-simulator`: `Error (Xcode): Unsupported mach-o filetype (only MH_OBJECT and MH_DYLIB can be linked) in <pkg>.framework/<pkg>` (a CMake/scikit-build extension is an MH_BUNDLE)
@@ -765,6 +774,20 @@ version-suspicious paths is the signature.
765774

766775
---
767776

777+
### setuptools: `ModuleNotFoundError: No module named 'distutils'` from a `byte_compile` script, after the extension already compiled
778+
779+
**Cause:** the sdist's `setup.cfg` sets `[install] optimize = 1`. `install_lib` then writes
780+
`.opt-1.pyc` files by running a generated script — `from distutils.util import byte_compile` —
781+
in a child interpreter, the cross venv's `python3.12`. Python 3.12 has no stdlib `distutils`,
782+
and setuptools' shim is not active in that child, so the build dies at the very end:
783+
`error: command '.../cross/bin/python3.12' failed with exit code 1`. The native part had
784+
already succeeded, which makes it look like a packaging bug rather than a config line.
785+
786+
**Fix:** patch the `[install] optimize` lines out of `setup.cfg`. Optimized bytecode is dead
787+
weight on a phone anyway. thrift (`recipes/thrift/patches/mobile.patch`).
788+
789+
---
790+
768791
### setuptools: `could not create 'build/lib...': Not a directory` (macOS local build)
769792

770793
**Cause:** the source root carries a Bazel **`BUILD`** file, which collides with

‎.claude/skills/local-recipe-testing/SKILL.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
name: local-recipe-testing
3-
description: Run a mobile-forge recipe's wheel ON-DEVICE locally — Android emulator and/or iOS simulator — instead of waiting ~1 hour for a CI mobile-test cycle. Covers the recipe-tester app loop (build wheel → stage → flet build → install → read console.log), and the non-obvious gotchas that each cost a wasted cycle: use forge's stripped dist/ wheel, build the recipe against the SAME Python flet bundles, clear flet's build cache between rebuilds, use a rootable (google_apis, not playstore) arm64 AVD to read the app-private console.log (it's in the app's cache/ dir), give the emulator enough RAM/disk, build ALL THREE iOS slices before `flet build ios-simulator`, use explicit simulator UDIDs when more than one sim is booted, verify the staged-test COUNT so a silently-failed staging can't replay stale tests as false passes, and check the built iOS `.app` actually carries your package (a failed site-packages sync still exits 0). Also covers forge slice syntax, bundling model assets next to recipe tests, test-only deps via the meta.yaml test.requires field, desktop pre-validation via a sys.modules alias shim, and consumer verify-apps for beyond-pytest validation. USE THIS SKILL when iterating on a recipe's on-device behaviour (import works? functions run? crashes?), reproducing or debugging a CI mobile-test failure locally, or whenever someone says the CI mobile test is too slow to iterate on. Sibling of `new-mobile-recipe` (authoring), `forge-ci` (CI runs), `forge-error-catalogue` (build errors), and `native-recipe-bumps` (version bumps); this one is specifically the fast on-device validation loop. macOS + Apple Silicon assumed (the host this was developed on).
3+
description: Run a mobile-forge recipe's wheel ON-DEVICE locally — Android emulator and/or iOS simulator — instead of waiting ~1 hour for a CI mobile-test cycle. Covers the recipe-tester app loop (build wheel → stage → flet build → install → read console.log), and the non-obvious gotchas that each cost a wasted cycle: use forge's stripped dist/ wheel, build the recipe against the SAME Python flet bundles, clear flet's build cache between rebuilds, use a rootable (google_apis, not playstore) arm64 AVD to read the app-private console.log (it's in the app's cache/ dir), give the emulator enough RAM/disk, build ALL THREE iOS slices before `flet build ios-simulator`, use explicit simulator UDIDs when more than one sim is booted, verify the staged-test COUNT so a silently-failed staging can't replay stale tests as false passes, check the built iOS `.app` actually carries your package (a failed site-packages sync, or a second iOS build running at the same time, still exits 0), and finish an iOS build by hand when Xcode 27 rejects flet's 13.0 deployment target. Also covers forge slice syntax, bundling model assets next to recipe tests, test-only deps via the meta.yaml test.requires field, desktop pre-validation via a sys.modules alias shim, and consumer verify-apps for beyond-pytest validation. USE THIS SKILL when iterating on a recipe's on-device behaviour (import works? functions run? crashes?), reproducing or debugging a CI mobile-test failure locally, or whenever someone says the CI mobile test is too slow to iterate on. Sibling of `new-mobile-recipe` (authoring), `forge-ci` (CI runs), `forge-error-catalogue` (build errors), and `native-recipe-bumps` (version bumps); this one is specifically the fast on-device validation loop. macOS + Apple Silicon assumed (the host this was developed on).
44
---
55

66
# Testing a mobile-forge recipe locally
@@ -204,6 +204,8 @@ carries every control value: `xcrun simctl spawn "$UDID" log show --last 2m --st
204204
```
205205
The known cause is an extension linked without `-Wl,-headerpad_max_install_names` (see the `forge-error-catalogue` skill), but the check is cheap and catches the whole class. The Android twin is gotcha #12's `unzip -l build/apk/…`.
206206

207+
**Two iOS builds running at once cause the same symptom.** Both stage into that one shared `dist_ios`, so whichever finishes second can bundle the other app's files (seen 2026-10-01: a recipe-tester `.app` carrying a consumer app's `main.pyc`). If anything else on the machine might be building for iOS — another terminal, another agent — give each build its own `PUB_CACHE=<dir>` (the first run re-downloads the pub packages), and pass the same `PUB_CACHE` to any manual `flutter build` that follows.
208+
207209
## Model assets & test-only deps
208210

209211
`stage_recipe.sh` copies **every** file in `recipes/<pkg>/tests/` into the app (`cp -r tests/. recipe_tests/`), so a model dropped next to the test file becomes an app asset. Two tiers:

‎.claude/skills/new-mobile-recipe/SKILL.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ If any phase fails, fix forward in that phase before moving on. Don't skip ahead
3131
flet iOS runtime reports as `"iOS"` → empty `get_adapters()`; one-line patch, setup.py-only
3232
sdist so `requirements.build: [setuptools]`). Same shadowing mechanics as pysodium.
3333
- **Pure-Python but SDIST-ONLY packages** (no wheel at all, but the sdist contains no native source — e.g. insightface's pure-Python releases). Still no recipe: the app opts in with `[tool.flet] source_packages = ["<name>"]` in its pyproject.toml, which makes `flet build` set `SERIOUS_PYTHON_ALLOW_SOURCE_DISTRIBUTIONS` and pip builds the sdist for the target. A recipe is only for packages that need *cross-compilation*.
34+
**"No native source" must be literal — an OPTIONAL extension does not count.** `source_packages` becomes `pip --no-binary <name>`, and pip's isolated PEP 517 build replaces `PYTHONPATH`, so serious_python's platform-faking `sitecustomize` is gone inside `setup.py`: it runs as the build host, the host compiler builds the "optional" extension for macOS, and pip installs that `cpXY-macosx_*` wheel into every mobile `--target` leg (it never tag-checks a wheel it built itself). The upstream pure-Python fallback never fires, because nothing failed. The app then works by accident — the foreign `.so` can't load, so the package falls back at import time — while shipping the Mach-O (Android: `lib/<abi>/lib<pkg>-<mod>.so`, Gradle only warns "Unable to strip"; iOS: a `.fwork` pointing at a framework that was never built). Code that retries the import per call pays a failed `dlopen` each time (thrift's `*Accelerated` protocols ran 12–23x slower than the plain classes). Such a package is a recipe candidate (`recipes/thrift/`); until it has one, `CXX=false`/`CC=false` in the `flet build` environment makes the fallback fire for real. Once a recipe wheel exists, consumers must drop the `source_packages` entry — `--no-binary` keeps forcing the source build.
3435
- **Bumping an existing `flet-lib*` recipe to a new upstream version** — use the sibling skill `native-recipe-bumps` instead. It encodes the version-conditional Jinja patterns specific to that workflow.
3536
- **Debugging a build failure in an existing recipe** — that's a different shape. This skill focuses on creation; for fixing, inspect `errors/<pkg>-*.log` directly and cross-reference the `forge-error-catalogue` skill.
3637

@@ -49,6 +50,7 @@ PyPI tags?
4950
│ │ `[tool.flet] source_packages` (see
5051
│ │ "When NOT to use" above)
5152
│ └── sdist has native source → CANDIDATE — same compile path
53+
│ (even an OPTIONAL extension: source_packages would build it for the host)
5254
└── no sdist either (wheels-only) → CANDIDATE — set `source.url` to a GitHub
5355
release/tag tarball (e.g. pyzbar). forge's
5456
PythonPackageBuilder honors source.url
@@ -70,6 +72,7 @@ Match the package to one of these shapes. Each maps to a template in `templates/
7072
|--------------------------------------------|---------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------|
7173
| Minimal C-extension, well-behaved upstream | C/C++ source, plain setuptools, no system deps | `templates/meta-minimal.yaml` |
7274
| Minimal C-ext + Android C++ runtime | Same as above but C++ source (bundled or upstream) → needs libc++_shared on Android | `templates/meta-minimal-with-android-libcpp.yaml` |
75+
| Optional C/C++ extension (setup.py falls back to pure Python on a compile error) | `Extension(optional=True)` (tornado) or a `build_ext` subclass that catches `CompileError` and re-runs `setup()` without `ext_modules` (thrift) — a failed cross-compile then ships a green pure-Python wheel | The minimal template + a `script_env` knob upstream already honors to make the failure fatal (thrift `CIBUILDWHEEL: "1"`, tornado `TORNADO_EXTENSION: "1"`), else a fail-loud patch (zeroconf). Copy `recipes/thrift/`. Tests must prove the `.so` loaded, and must FAIL when run against a pure build (`CC=false CXX=false pip install --no-binary <pkg>`) — that negative control is what shows the suite cannot pass on a fallback |
7376
| Rust via PyO3 / maturin | `[build-system].requires` contains `maturin` or `setuptools-rust` | `templates/meta-rust.yaml` |
7477
| C-ext needing patches | Upstream `setup.py` reaches for host `/usr/include`, has `if sys.platform` branches, or hardcodes paths | `templates/meta-with-patches.yaml` + patch file in `patches/` |
7578
| meson-python build backend | `[build-system].build-backend = "mesonpy"` (scipy, scikit-image, scikit-learn, pywavelets, contourpy) | No template — copy from `recipes/pywavelets/` (on main): `build.backend-args:` list with `-Csetup-args=--cross-file` + `-Csetup-args={MESON_CROSS_FILE}`; extra meson options ride the same way (`-Csetup-args=-D<opt>=<val>`) |

0 commit comments

Comments
 (0)