Skip to content

fix: add Android support #1142

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

fix: add Android support #1142

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
66386b1
Add Android support
mhsmith Sep 3, 2025
869c9c1
Improve wording
mhsmith Sep 5, 2025
cbc124d
Revert removal of Interpreter
mhsmith Sep 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ An example `CMakeLists.txt`:
cmake_minimum_required(VERSION 3.15...3.30)
project(${SKBUILD_PROJECT_NAME} LANGUAGES C)

find_package(Python COMPONENTS Interpreter Development.Module REQUIRED)
find_package(Python COMPONENTS Development.Module REQUIRED)
Copy link
Collaborator

@LecrisUT LecrisUT Sep 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skipping interpreter can be good and bad. Iirc cross compilation with cross-env behaves differently with and without it, but I don't remember which one is more preferred.

Copy link
Collaborator

@henryiii henryiii Sep 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think CMake 4.1 kind of forces the no Interpreter case. From what I understand, you can set a emulated Python for it, though.

Copy link
Collaborator

@henryiii henryiii Sep 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, since we don't correctly find and pass the Python library, adding the Interpreter components helps on Windows. If we could reliably find the Python library, then we don't need the Interpreter component. For some reason CMake is better than us at finding Python. I'd like a more principled approach than #1093's seeming "try everything" method, but we likely need something to get rid of the Interpreter component.

Copy link
Collaborator

@LecrisUT LecrisUT Sep 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally think the user should provide the hints themselves when cross-compiling in the toolchain file, and we just document that. There are too many edge-cases we would have to account for, which otherwise could be better handled by SYSROOT and such

Copy link
Collaborator

@henryiii henryiii Sep 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm referring to when not cross compiling - we currently should compute this unless the user specifies something different, but instead we usually don't find it and rely on CMake to find the Python library using the interpreter. Yes, when cross-compiling, it has to be specified.


Python_add_library(_module MODULE src/module.c WITH_SOABI)
install(TARGETS _module DESTINATION ${SKBUILD_PROJECT_NAME})
Expand Down
2 changes: 1 addition & 1 deletion docs/examples/downstream/nanobind_example/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ endif()
# Try to import all Python components potentially needed by nanobind
find_package(
Python 3.8 REQUIRED
COMPONENTS Interpreter Development.Module
COMPONENTS Development.Module
OPTIONAL_COMPONENTS Development.SABIModule)

# Import nanobind through CMake's find_package mechanism
Expand Down
2 changes: 1 addition & 1 deletion docs/examples/downstream/pybind11_example/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ project(
VERSION ${SKBUILD_PROJECT_VERSION}
LANGUAGES CXX)

find_package(Python REQUIRED COMPONENTS Interpreter Development.Module)
find_package(Python REQUIRED COMPONENTS Development.Module)
find_package(pybind11 CONFIG REQUIRED)

python_add_library(_core MODULE src/main.cpp WITH_SOABI)
Expand Down
2 changes: 1 addition & 1 deletion docs/examples/getting_started/abi3/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ project(${SKBUILD_PROJECT_NAME} LANGUAGES C)

find_package(
Python
COMPONENTS Interpreter Development.SABIModule
COMPONENTS Development.SABIModule
REQUIRED)

python_add_library(example MODULE example.c WITH_SOABI USE_SABI 3.8)
Expand Down
Binary file added .../examples/getting_started/c/wheelhouse/example-0.0.1-cp313-cp313-android_21_arm64_v8a.whl
View file
Open in desktop
Binary file not shown.
2 changes: 1 addition & 1 deletion docs/examples/getting_started/fortran/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ project(${SKBUILD_PROJECT_NAME} LANGUAGES C Fortran)

find_package(
Python
COMPONENTS Interpreter Development.Module NumPy
COMPONENTS Development.Module NumPy
REQUIRED)

# F2PY headers
Expand Down
2 changes: 1 addition & 1 deletion docs/examples/getting_started/nanobind/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
cmake_minimum_required(VERSION 3.15...3.26)
project(${SKBUILD_PROJECT_NAME} LANGUAGES CXX)

find_package(Python 3.8 REQUIRED COMPONENTS Interpreter Development.Module)
find_package(Python 3.8 REQUIRED COMPONENTS Development.Module)

find_package(nanobind CONFIG REQUIRED)

Expand Down
2 changes: 1 addition & 1 deletion docs/examples/getting_started/swig/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ project(${SKBUILD_PROJECT_NAME} LANGUAGES C)

find_package(
Python
COMPONENTS Interpreter Development.Module
COMPONENTS Development.Module
REQUIRED)

find_package(
Expand Down
12 changes: 6 additions & 6 deletions docs/guide/cmakelists.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,13 @@ Scikit-build-core provides several useful variables:
You can directly use FindPython:

```cmake
find_package(Python COMPONENTS Interpreter Development.Module REQUIRED)
find_package(Python COMPONENTS Development.Module REQUIRED)
```

You always want to find at least `Interpreter` and the `Module` component of the
`Development` package. You do not want to find the entire `Development` package,
as that include `Embed` component, which is not always present and is not
related to making Python extension modules.
You always want to find at least the `Development.Module` component. You do not
want to find the entire `Development` component, as that includes
`Development.Embed`, which is not always present and is not related to making
Python extension modules.

If you are making a Limited API / Stable ABI package, you'll need the
`Development.SABIModule` component instead (CMake 3.26+). You can use the
Expand Down Expand Up @@ -129,7 +129,7 @@ When you do that, `${SKBUILD_SABI_COMPONENT}` will be set to
remain an empty string otherwise (PyPy). This allows the following idiom:

```cmake
find_package(Python REQUIRED COMPONENTS Interpreter Development.Module ${SKBUILD_SABI_COMPONENT})
find_package(Python REQUIRED COMPONENTS Development.Module ${SKBUILD_SABI_COMPONENT})
```

This will require the `Development.SABIModule` component only if
Expand Down
15 changes: 14 additions & 1 deletion docs/guide/crosscompile.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ correct suffix. These values are set by cibuildwheel when cross-compiling.

It should be possible to cross-compile to Linux, but due to the challenges of
getting the manylinux RHEL devtoolkit compilers, this is currently a TODO. See
`py-build-cmake <https://tttapa.github.io/py-build-cmake/Cross-compilation.html>`\_
[py-build-cmake](https://tttapa.github.io/py-build-cmake/Cross-compilation.html)
for an alternative package's usage of toolchain files.

### Intel to Emscripten (Pyodide)
Expand All @@ -64,3 +64,16 @@ by setting `_PYTHON_SYSCONFIGDATA_NAME`. This causes values like `SOABI` and
This is unfortunately incorrectly stripped from the cmake wrapper pyodide uses,
so FindPython will report the wrong values, but pyodide-build will rename the
.so's afterwards.

## Android

To build for Android, you'll need the following items, all of which will be
provided automatically if you use cibuildwheel:

- An Android
[toolchain file](https://cmake.org/cmake/help/latest/variable/CMAKE_TOOLCHAIN_FILE.html)
which adds the location of the Python headers and libraries to
`CMAKE_FIND_ROOT_PATH`.
Copy link
Collaborator

@LecrisUT LecrisUT Sep 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Until #1050 is resolved, better add a note about how to pass CMAKE_TOOLCHAIN_FILE. Also should add an explicit note about what happens if you are finding Interpreter component or not.

Copy link
Author

@mhsmith mhsmith Sep 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added more details of the toolchain file, and reverted all the Interpreter changes as explained in my comment below.

- Compiler paths and flags, either in the toolchain file or in environment
variables.
- A Python environment which simulates Android.
16 changes: 8 additions & 8 deletions docs/guide/faqs.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,14 +42,14 @@ is missing some value you need, please open an issue and let us know.

## Finding Python

One common mistake when using FindPython is to forget to only request the
`Development.Module` component. If you request `Development`, you will also
require the `Development.Embed` component, which will require the Python
libraries to be found for linking. When building a module on Unix, you do not
link to Python - the Python symbols are already loaded in the interpreter.
What's more, the manylinux image (which is used to make redistributable Linux
wheels) does not have the Python libraries, both to avoid this mistake, and to
reduce size.
One common mistake when using FindPython is to request the `Development`
component, which requires the `Development.Embed` component, which will require
the Python libraries to be found for linking. When building a module on Unix,
you do not link to Python - the Python symbols are already loaded in the
interpreter. What's more, the manylinux image (which is used to make
redistributable Linux wheels) does not have the Python libraries, both to avoid
this mistake, and to reduce size. Instead, you should only request the
`Development.Module` component.

## Cross compiling

Expand Down
2 changes: 1 addition & 1 deletion docs/guide/migration_guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ python_extension_module(${LIBRARY})
to

```cmake
find_package(Python COMPONENTS Interpreter Development.Module REQUIRED)
find_package(Python COMPONENTS Development.Module REQUIRED)
python_add_library(${LIBRARY} MODULE WITH_SOABI ${FILENAME})
```

Expand Down
12 changes: 7 additions & 5 deletions src/scikit_build_core/builder/builder.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -254,11 +254,13 @@ def configure(
cache_config[f"{prefix}_ROOT_DIR"] = Path(sys.prefix)
cache_config[f"{prefix}_INCLUDE_DIR"] = python_include_dir
cache_config[f"{prefix}_FIND_REGISTRY"] = "NEVER"
# FindPython may break if this is set - only useful on Windows
if python_library and sysconfig.get_platform().startswith("win"):
cache_config[f"{prefix}_LIBRARY"] = python_library
if python_sabi_library and sysconfig.get_platform().startswith("win"):
cache_config[f"{prefix}_SABI_LIBRARY"] = python_sabi_library
# Setting the library location is only needed on some platforms - on
# other platforms it may break FindPython.
if sysconfig.get_platform().startswith(("win", "android")):
if python_library:
cache_config[f"{prefix}_LIBRARY"] = python_library
if python_sabi_library:
cache_config[f"{prefix}_SABI_LIBRARY"] = python_sabi_library
Copy link
Collaborator

@LecrisUT LecrisUT Sep 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part needs some discussion. I think this would point these to an unwanted location. Could really use a more concrete example of how to cross-compile to see how these are expanded. #1050 is also relevant for these.

Copy link
Collaborator

@henryiii henryiii Sep 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what we already have, though, just with android added alongside windows?

Copy link
Collaborator

@LecrisUT LecrisUT Sep 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but I don't believe it is needed or that it has the intended effect when cross-compiling. E.g. are we passing the library of the host or target environment there? A more concrete example of how a cross-compiled android project would help a lot, as well as how each environment does it

Copy link
Collaborator

@henryiii henryiii Sep 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't pass it, then it is required to include Interpreter, since CMake has to find it, it does so by queuing the Python interpreter (which is set to sys.interpreter, so literally the same python we are in now). If a user provides a toolchain file for cross compiling, these get overridden.

Though, to your point, since android will never be the host, not sure adding it here is needed?

Copy link
Collaborator

@LecrisUT LecrisUT Sep 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't pass it, then it is required to include Interpreter, since CMake has to find it, it does so by queuing the Python interpreter (which is set to sys.interpreter, so literally the same python we are in now).

That is specific to windows though right? Because on linux it would use the bash file (forgot the name of it) that contains the configure values used when compiling (dunno why they don't provide something similar to windows though).

Though, to your point, since android will never be the host, not sure adding it here is needed?

Yes that would be preferred imo with relevant documentation

Copy link
Collaborator

@henryiii henryiii Sep 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not needed on other platforms, only Windows and Android care. So I'm not really sure, as it's not used.

if numpy_include_dir:
cache_config[f"{prefix}_NumPy_INCLUDE_DIR"] = numpy_include_dir

Expand Down
12 changes: 11 additions & 1 deletion tests/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import sysconfig
from importlib import metadata
from pathlib import Path
from typing import Any, Literal, overload
from typing import TYPE_CHECKING, Any, Literal, overload

import virtualenv as _virtualenv

Expand All @@ -19,6 +19,8 @@
else:
import tomllib

if TYPE_CHECKING:
from pytest_subprocess import FakeProcess

import pytest
from packaging.requirements import Requirement
Expand All @@ -30,6 +32,14 @@
VIRTUALENV_VERSION = Version(metadata.version("virtualenv"))


@pytest.fixture
def fp(fp: FakeProcess) -> FakeProcess:
# For program_search._macos_binary_is_x86
fp.register(["lipo", fp.any()], returncode=1)
fp.register(["file", fp.any()], returncode=1)
return fp


@pytest.fixture(scope="session")
def pep518_wheelhouse(tmp_path_factory: pytest.TempPathFactory) -> Path:
wheelhouse = tmp_path_factory.mktemp("wheelhouse")
Expand Down
2 changes: 1 addition & 1 deletion tests/packages/abi3_pyproject_ext/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ project(

find_package(
Python
COMPONENTS Interpreter Development.Module ${SKBUILD_SABI_COMPONENT}
COMPONENTS Development.Module ${SKBUILD_SABI_COMPONENT}
REQUIRED)

if(NOT "${SKBUILD_SABI_VERSION}" STREQUAL "")
Expand Down
2 changes: 1 addition & 1 deletion tests/packages/abi3_setuptools_ext/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ project(

find_package(
Python
COMPONENTS Interpreter Development.SABIModule
COMPONENTS Development.SABIModule
REQUIRED)

python_add_library(abi3_example MODULE abi3_example.c WITH_SOABI USE_SABI 3.8)
Expand Down
2 changes: 1 addition & 1 deletion tests/packages/broken_fallback/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ endif()

find_package(
Python
COMPONENTS Interpreter Development.Module
COMPONENTS Development.Module
REQUIRED)

python_add_library(example MODULE main.c WITH_SOABI)
Expand Down
2 changes: 1 addition & 1 deletion tests/packages/dynamic_metadata/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ project(
LANGUAGES C
VERSION ${SKBUILD_PROJECT_VERSION})

find_package(Python COMPONENTS Interpreter Development.Module)
find_package(Python COMPONENTS Development.Module)
set(Python_SOABI ${SKBUILD_SOABI})

python_add_library(_module MODULE src/module.c WITH_SOABI)
Expand Down
2 changes: 1 addition & 1 deletion tests/packages/importlib_editable/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ project(${SKBUILD_PROJECT_NAME} LANGUAGES C)

find_package(
Python
COMPONENTS Interpreter Development.Module
COMPONENTS Development.Module
REQUIRED)

python_add_library(emod MODULE emod.c WITH_SOABI)
Expand Down
2 changes: 1 addition & 1 deletion tests/packages/navigate_editable/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ project(
LANGUAGES C
VERSION ${SKBUILD_PROJECT_VERSION})

find_package(Python COMPONENTS Interpreter Development.Module)
find_package(Python COMPONENTS Development.Module)

python_add_library(c_module MODULE src/shared_pkg/c_module.c WITH_SOABI)

Expand Down
2 changes: 1 addition & 1 deletion tests/packages/simple_pyproject_ext/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ project(

find_package(
Python
COMPONENTS Interpreter Development.Module
COMPONENTS Development.Module
REQUIRED)

find_package(pybind11 CONFIG REQUIRED)
Expand Down
2 changes: 1 addition & 1 deletion tests/packages/simple_pyproject_source_dir/src/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ project(

find_package(
Python
COMPONENTS Interpreter Development.Module
COMPONENTS Development.Module
REQUIRED)

find_package(pybind11 CONFIG REQUIRED)
Expand Down
2 changes: 1 addition & 1 deletion tests/packages/simplest_c/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ project(
LANGUAGES C
VERSION ${SKBUILD_PROJECT_VERSION})

find_package(Python COMPONENTS Interpreter Development.Module)
find_package(Python COMPONENTS Development.Module)

python_add_library(_module MODULE src/module.c WITH_SOABI)

Expand Down