Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linker library name format in setup.py #458

Open
evertrol opened this issue Dec 4, 2024 · 0 comments
Open

Linker library name format in setup.py #458

evertrol opened this issue Dec 4, 2024 · 0 comments

Comments

@evertrol
Copy link
Contributor

evertrol commented Dec 4, 2024

setup.py has the following if-else block (twice):

            if 'clang' in os.environ['CC']:
                libraries += ['inversion.so', 'deflection.so']
            else:
                libraries += [':inversion.so', ':deflection.so']

The : indicates to the GNU linker that the library name should be taken as is, that is, not prepended with lib1.
The linker on Darwin (macOS), probably a BSD(-derived) linker, takes any(?) name that has a extension as the literal name, and won't search for lib<name>.dylib or lib<name>a2. So that linker does automagically, whereas the GNU linker wants to be explicit.

Note the use of linker in the above paragraph: it is not the compiler that makes the difference, but the linker. On the command line, it will look like the compiler (the linking step still shows the compiler being used), but under the hood it calls the linker, ld. Therefore, the linker should be the differentiating factor which of the two if-branches is used, not the compiler (or its environment variable, CC).

This issue pops when building X-PSI on macOS with gcc (note: not the built-in gcc, which is clang in disguise: check gcc --version; but for example gcc installed with Homebrew), or when using clang on Linux. Both builds will error at this point when linking with the so-files is done.

A manual, local, quick fix is just to insert or remove the colons in front of the library names in setup.py, and the build can continue. But this is only a local solution, and can't be implemented in setup.py by default.

The linker, however, is trickier to determine than the compiler. But the linker is also tied into the system ecosystem (through system libraries); for example, the GNU linker is not supported on Darwin (though perhaps a BSD linker can be installed on a Linux system).
Instead, one could try and determine the system OS and use that, instead of the compiler, in the if-statement: sys.platform may be of use here.

Note that the use cases will be rare: the default compiler on macOS tends to be clang (installed as part of the XCode tools), and even the default gcc is just clang in disguise. On Linux, the majority of times the default compiler is gcc. I don't know which linker the Intel compiler will use; perhaps the GNU linker, perhaps it has its own linker (but given that the OS is formally GNU/Linux, I suspect the linker is the GNU one). Most people are not likely to deviate from this, so the check for clang tends to nearly always be True for macOS and False for Linux.

I'll leave this issue here for cases where someone does run into it, or in case someone does really wants to fix this. The latter probably requires checking builds on various clusters, including Intel compilers, to which I have no access.


Footnotes

  1. from the GNU ld manpage:

    -l namespec
    --library=namespec
    Add the archive or object file specified by namespec to the list of files to link. This option may be used any number of times. If namespec is of the form :filename, ld will search the library path for a file called filename, otherwise it will search the library path for a file called libnamespec.a.

  2. from the Darwin ld manpage:

    -lx This option tells the linker to search for libx.dylib or libx.a in the library search path. If string x is of the form y.o, then that file is searched for in the same places, but without prepending lib' or appending .a' or `.dylib' to the filename.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant