f2py entry point with non-functioning shebang in osx-arm cross-compile

[bnavigator]

Solution to issue cannot be found in the documentation.

• I checked the documentation.

Issue

With

requirements:
  build: 
    - numpy                   # [build_platform != target_platform]

for the osx-arm cross-compilation, calling f2py3 fails with:

/Users/runner/miniforge3/conda-bld/slycot_1657493761007/_build_env/bin/f2py3: line 3: import: command not found
/Users/runner/miniforge3/conda-bld/slycot_1657493761007/_build_env/bin/f2py3: line 4: import: command not found
from: can't read /var/mail/numpy.f2py.f2py2e
/Users/runner/miniforge3/conda-bld/slycot_1657493761007/_build_env/bin/f2py3: line 7: syntax error near unexpected token `('
/Users/runner/miniforge3/conda-bld/slycot_1657493761007/_build_env/bin/f2py3: line 7: `    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])'

$> which f2py3
$BUILD_PREFIX/bin/f2py3
$> head f2py3
#!$BUILD_PREFIX/bin/python
# -*- coding: utf-8 -*-
import re
import sys
from numpy.f2py.f2py2e import main

See discussion in conda-forge/slycot-feedstock#50 and conda-forge/slycot-feedstock#49.

cc @conda-forge/help-osx-arm64

Installed packages

bzip2:           1.0.8-h3422bc3_4              conda-forge
    c-ares:          1.18.1-h3422bc3_0             conda-forge
    ca-certificates: 2022.6.15-h4653dfc_0          conda-forge
    cmake:           3.23.2-h52850c7_0             conda-forge
    distro:          1.6.0-pyhd8ed1ab_0            conda-forge
    expat:           2.4.8-h6b3803e_0              conda-forge
    krb5:            1.19.3-he492e65_0             conda-forge
    libblas:         3.9.0-5_h880f123_netlib       conda-forge
    libcblas:        3.9.0-5_h880f123_netlib       conda-forge
    libcurl:         7.83.1-h7965298_0             conda-forge
    libcxx:          14.0.6-h04bba0f_0             conda-forge
    libedit:         3.1.20191231-hc8eb9b7_2       conda-forge
    libev:           4.33-h642e427_1               conda-forge
    libffi:          3.4.2-h3422bc3_5              conda-forge
    libgfortran:     5.0.0.dev0-11_0_1_hf114ba7_23 conda-forge
    libgfortran5:    11.0.1.dev0-hf114ba7_23       conda-forge
    liblapack:       3.9.0-5_h880f123_netlib       conda-forge
    libnghttp2:      1.47.0-hf30690b_0             conda-forge
    libssh2:         1.10.0-h7a5bd25_2             conda-forge
    libuv:           1.43.0-h3422bc3_0             conda-forge
    libzlib:         1.2.12-ha287fd2_1             conda-forge
    llvm-openmp:     14.0.4-hd125106_0             conda-forge
    lz4-c:           1.9.3-hbdafb3b_1              conda-forge
    make:            4.3-he57ea6c_1                conda-forge
    ncurses:         6.3-h07bb92c_1                conda-forge
    numpy:           1.23.1-py310h0a343b5_0        conda-forge
    openssl:         3.0.5-ha287fd2_0              conda-forge
    packaging:       21.3-pyhd8ed1ab_0             conda-forge
    pip:             22.1.2-pyhd8ed1ab_0           conda-forge
    pyparsing:       3.0.9-pyhd8ed1ab_0            conda-forge
    python:          3.10.5-h4eee789_0_cpython     conda-forge
    python_abi:      3.10-2_cp310                  conda-forge
    readline:        8.1.2-h46ed386_0              conda-forge
    rhash:           1.4.3-he4db4b2_0              conda-forge
    scikit-build:    0.15.0-pyhb871ab6_0           conda-forge
    setuptools:      63.1.0-py310hbe9552e_0        conda-forge
    sqlite:          3.39.0-h40dfcc0_0             conda-forge
    tk:              8.6.12-he1e0b03_0             conda-forge
    tzdata:          2022a-h191b570_0              conda-forge
    wheel:           0.37.1-pyhd8ed1ab_0           conda-forge
    xz:              5.2.5-h642e427_1              conda-forge
    zlib:            1.2.12-ha287fd2_1             conda-forge
    zstd:            1.5.2-hd705a24_2              conda-forge


    bzip2:                       1.0.8-h0d85af4_4          conda-forge
    c-ares:                      1.18.1-h0d85af4_0         conda-forge
    ca-certificates:             2022.6.15-h033912b_0      conda-forge
    cctools_osx-64:              973.0.1-h3eff9a4_10       conda-forge
    cctools_osx-arm64:           973.0.1-h98580c8_10       conda-forge
    clang:                       13.0.1-h694c41f_0         conda-forge
    clang-13:                    13.0.1-default_he082bbe_0 conda-forge
    clang_osx-arm64:             13.0.1-hdd0e76e_2         conda-forge
    clangxx:                     13.0.1-default_he082bbe_0 conda-forge
    cmake:                       3.23.2-hf2c7296_0         conda-forge
    compiler-rt:                 13.0.1-he01351e_0         conda-forge
    compiler-rt_osx-64:          13.0.1-hd3f61c9_0         conda-forge
    cross-python_osx-arm64:      3.10-27_cpython           conda-forge
    crossenv:                    1.2.0-pyhd8ed1ab_7        conda-forge
    distro:                      1.6.0-pyhd8ed1ab_0        conda-forge
    expat:                       2.4.8-h96cf925_0          conda-forge
    gettext:                     0.19.8.1-hd1a6beb_1008    conda-forge
    gfortran_impl_osx-64:        9.3.0-h9cc0e5e_23         conda-forge
    gfortran_impl_osx-arm64:     11.0.1.dev0-h3653101_23   conda-forge
    gfortran_osx-arm64:          11.0.1.dev0-h64a2375_15   conda-forge
    gmp:                         6.2.1-h2e338ed_0          conda-forge
    isl:                         0.22.1-hb1e8313_2         conda-forge
    krb5:                        1.19.3-hb98e516_0         conda-forge
    ld64_osx-64:                 609-h6fbe7a8_10           conda-forge

Environment info

(conda-forge osx-arm build on azure)

[h-vetinari]

Thanks for the report @bnavigator. Seems there's a bunch more relevant info in conda-forge/slycot-feedstock#50.

@isuruf @beckermr @xhochy
How are entry-points handled/supported in cross-compilation (if at all)? Seems like the current f2py entry point doesn't like it to be called while cross-compiling for osx-arm.

[beckermr]

@erykoff may have insight here too

[bnavigator]

The explanation in conda-forge/slycot-feedstock#50 (comment) doesn't look plausible to me.

[beckermr]

Did you try the suggested change? conda-build won't know about entry points unless you tell it about them. It may be doing special handing for them for cross-compiling (and does for some cases like noarch builds already).

[h-vetinari]

I overlooked the # [win] selector when checking that the entry-point is defined, apologies.

[bnavigator]

• How am I supposed to change the numpy-feedstock for tests in my own feedstock?
• There is nothing in the documentation about regular entry_points (unless for noarch packages). Since when is there a requirement to duplicate the python package entry point specification in the recipe?

[beckermr]

1. You can build the numpy version/recipe you need locally and try cross-compiling the recipe against the local numpy package with the change.

2. I am not claiming there is a requirement but merely postulating that there might be one for cross-compiling. There may not be. I don't know.

As it seems easy to test this out, it seems like a worthwhile step.

FYI, both Isuru and Uwe are either out of contact completely or rarely in contact for at least a few months so you may not get help from them right away. They both know more than I do about this so we may have to wait to get this resolved.

[bnavigator]

Negative. I don't have a mac to test this locally.

[bnavigator]

Open unanswered question: conda/conda-build#4291

Knowledgebase about noarch entry_points, unclear about what is needed on non-noarch: https://conda-forge.org/docs/maintainer/knowledge_base.html#noarch-python

[beckermr]

Ahhh then we'll have to wait until someone who does have a mac has the time or someone more knowledgeable can confirm what change we need. :/

[h-vetinari]

What's the harm in building this feedstock without the # [win] selector for the entry-point? The tests contain the CLI call to f2py.

And even if we don't feel comfortable in doing that, we could push the builds to a dev label, for example.

[beckermr]

Good find on that conda-build issue! Many of the core conda-build contributors are not active on the project anymore for various reasons and so that question might best be answered by grepping the code.

@h-vetinari Great idea! Using a dev label is totally fine. If you need me to make a branch in the upstream feedstock just bump in the PR to be merged to it.

[bnavigator]

What's the harm in building this feedstock without the # [win] selector for the entry-point? The tests contain the CLI call to f2py.

You would have to make an additional line for nowin (or whatever is the correct conditional) as the # [win] line adds a specific batch wrapper.

[beckermr]

You would have to make an additional line for nowin (or whatever is the correct conditional) as the # [win] line adds a specific batch wrapper.

Seems not so hard to do with a little time and elbow grease. Feel free to go ahead if you'd like.