Repository navigation
Conversation
This PR needs a
|
|
Hi @BicMak! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
|
@claude review this PR |
|
Claude finished @rascani's task in 4m 0s —— View job This PR works on a clean first build, but a few things need fixing first: switching to Detailed reviewPR Review: #24345SummaryThis PR adds A clean first build looks right. The main problem is re-running the script: switching to Code Quality
Infrastructure
Security
RecommendationRequest Changes The stale-state problem when adding or switching |
setup.sh builds with the Arm toolchain's bundled glibc (2.40), so the binaries require GLIBC_2.38, GLIBCXX_3.4.32 and CXXABI_1.3.15. Raspberry Pi OS Bookworm (glibc 2.36) fails to load them with "GLIBC_2.38 not found". --os <bookworm|trixie> builds against that Debian release's glibc, using an arm64 sysroot from mmdebstrap, and links libstdc++/libgcc statically. The toolchain's static libstdc++ references __isoc23_strtoul, which glibc < 2.38 lacks, so glibc_compat.c provides it only for older sysroots (--defsym cannot alias it because strtoul lives in a shared library). After the build, the required GLIBC version is checked against what the target OS ships. Without --os the behavior is unchanged and a warning points to the new option. Bullseye (glibc 2.31) is rejected: the libraries bundled with the Arm toolchain require glibc >= 2.35. Authored with Claude Code.
38ea848 to
b97f0bf
Compare
|
Thank you for the feedback on using --exclude-libs,ALL in the previous review. I did not include this change in the latest commit, and I thought it would be helpful to explain why. I've summarized my findings below and would appreciate your thoughts. SummaryAfter applying and testing
So the "root problem of libstdc++ being instantiated in both main and runner" does not occur in the current code, and the second submission left the link options unchanged. Details1. Background: why libstdc++ is statically linked
2. Criterion: when are two copies a problem?Having two copies of the code is not a problem by itself. It breaks when both copies must see the same global state but each sees its own.
3. Measurement: the current build shares a single libstdc++The link commands for
Verification script, result tables, logsVerification script (
|
| Step | Input | Output |
|---|---|---|
| 0 | Deployed binaries | sha256 and size (records which build was measured) |
| 1 | Both link.txt → relink |
Linker maps and unstripped binaries. The stripped relink's size is compared with the deployed binary to confirm they match |
| 2 | LOAD lines in llama_main.map |
Link order |
| 3 | Archive member lists in both maps | Number of libstdc++ objects in each binary |
| 4 | "Archive member included" in llama_main.map |
Why the exe pulled each object |
| 5, 6 | nm |
Symbols the exe imports from the .so, and where each key symbol is defined |
Result: libstdc++ objects pulled in
| Binary | libstdc++.a | libgcc_eh.a | libgcc.a |
|---|---|---|---|
libllama_runner.so |
111 | 5 | 16 |
llama_main |
2 (eh_ptr.o, condition_variable.o) |
5 | 12 |
Where key symbols are defined (T = defined and exported, t = defined but hidden, U = imported from elsewhere):
| Symbol | exe | .so | Exported by .so? |
|---|---|---|---|
__cxa_throw |
U | T | yes |
__cxa_begin_catch |
U | T | yes |
__cxa_get_globals |
U | T | yes |
__gxx_personality_v0 |
U | T | yes |
operator new(unsigned long) |
U | T | yes |
std::rethrow_exception(...) |
T | - | no |
std::condition_variable::wait(...) |
T | - | no |
_Unwind_RaiseException |
T | t | no |
_Unwind_Resume |
T | t | no |
Script log (excerpt)
==== 3. Runtime archive members pulled into each binary (from linker maps) ====
-- libllama_runner: libstdc++.a 111, libgcc_eh.a 5, libgcc.a 16
-- llama_main: libstdc++.a 2, libgcc_eh.a 5, libgcc.a 12
-- llama_main members (all):
libgcc.a(_divdc3.o)
libgcc.a(_divsc3.o)
libgcc.a(_muldc3.o)
libgcc.a(_mulsc3.o)
libgcc.a(cas_8_4.o)
libgcc.a(divtf3.o)
libgcc.a(floatunditf.o)
libgcc.a(ldadd_4_4.o)
libgcc.a(lse-init.o)
libgcc.a(sfp-exceptions.o)
libgcc.a(swp_8_4.o)
libgcc.a(trunctfdf2.o)
libgcc_eh.a(__aarch64_have_sme.o)
libgcc_eh.a(__arm_tpidr2_save.o)
libgcc_eh.a(__arm_za_disable.o)
libgcc_eh.a(unwind-dw2-fde-dip.o)
libgcc_eh.a(unwind-dw2.o)
libstdc++.a(condition_variable.o)
libstdc++.a(eh_ptr.o)
-- libllama_runner libgcc_eh members:
libgcc_eh.a(__aarch64_have_sme.o)
libgcc_eh.a(__arm_tpidr2_save.o)
libgcc_eh.a(__arm_za_disable.o)
libgcc_eh.a(unwind-dw2-fde-dip.o)
libgcc_eh.a(unwind-dw2.o)
-- llama_main runtime members that also exist in libllama_runner.so:
libgcc.a(cas_8_4.o)
libgcc.a(floatunditf.o)
libgcc.a(ldadd_4_4.o)
libgcc.a(lse-init.o)
libgcc.a(sfp-exceptions.o)
libgcc.a(swp_8_4.o)
libgcc_eh.a(__aarch64_have_sme.o)
libgcc_eh.a(__arm_tpidr2_save.o)
libgcc_eh.a(__arm_za_disable.o)
libgcc_eh.a(unwind-dw2-fde-dip.o)
libgcc_eh.a(unwind-dw2.o)
==== 6. Key symbols: where each is defined (T/t = defined, U = imported, - = absent; relinked unstripped binaries) ====
symbol exe so so-exported
__cxa_throw U T yes
__cxa_begin_catch U T yes
__cxa_get_globals U T yes
__gxx_personality_v0 U T yes
operator new(unsigned long) U T yes
operator delete(void*) - T yes
std::locale::classic() - T yes
std::rethrow_exception(std::__exception_ptr::exception_ptr) T - no
std::condition_variable::wait(std::unique_lock<std::mutex>&) T - no
_Unwind_RaiseException T t no
_Unwind_Resume T t no
_Unwind_SetGR T t no
dwarf_reg_size_table b b noWhy: link order + symbols exported by the .so
examples/models/llama/CMakeLists.txt:250:
target_link_libraries(llama_main PUBLIC llama_runner ${link_libraries})
The actual llama_main link command (CMakeFiles/llama_main.dir/link.txt):
$ tr ' ' '\n' < examples/models/llama/CMakeFiles/llama_main.dir/link.txt | grep -nE 'llama_runner|\.a$|static-lib'
16: -static-libstdc++
38: .../libxnnpack_backend.a # whole-archive: backend registration code goes into the exe
45: runner/libllama_runner.so # the .so comes first
46: .../libexecutorch.a ... # the .a files come after- The linker resolves symbols in order, and
libllama_runner.socomes beforelibstdc++.a. - The
.sohas no symbol-hiding option, so the libstdc++ symbols statically linked into it are exported as-is. - So the C++ runtime the exe needs is resolved from the
.sofirst, and only the 2 objects missing from the.soare pulled fromlibstdc++.a. - As a result, the libstdc++ actually used is the single copy inside the
.so.
4. --exclude-libs,ALL: crash
- Cause: all symbols of the
.soare hidden, so the exe gets its own copy of the backend registry. Backends register into the exe's registry, while lookups go to the.so's registry. - Result: on Pi 5 (Bookworm), the process exits with
backend not registered.
gdb log
executorch::runtime::Program const*, executorch::runtime::BackendInitContext&, executorch::runtime::BackendDelegate*) () from /home/jiwon/poc-exp/flag/libllama_runner.so
CHECK returned x0=0x0 (NULL: backend not registered)
[Inferior 1 (process 2015) exited with code 01]5. --exclude-libs,libstdc++.a: runs, but creates two copies
- Confirmed that it runs on Pi 5.
- However, isolating libstdc++ leaves one copy in the exe and one in the
.so: two copies (U __cxa_throwdisappears fromnm -D llama_main). - In other words, it does not fix the state the reviewer was concerned about; it creates it.
6. Scope and limitations
- This PR only changes
setup.shand the generated toolchain file. - Limitation 1: it depends on link order. Sharing a single libstdc++ comes from the
.sobeing linked before the.afiles. This order is not introduced by this PR; it is the existing upstream structure (examples/models/llama/CMakeLists.txt:250). - Limitation 2: the libgcc_eh unwinder already exists twice. libgcc_eh is built with hidden symbols in the archive itself, so the
.sodoes not export it (exeT,.sotin the symbol table in Section 3 details). However, the unwinder is designed to walk all loaded modules viadl_iterate_phdr(unwind-dw2-fde-dip.o) each time, with no shared state, so it can unwind across the exe and the.soeven with two copies. This is the intended behavior of-static-libgcc.
7. Conclusion
| Configuration | libstdc++ | Registry | Verdict |
|---|---|---|---|
| Current | One copy (shared) | One copy (shared) | Keep |
--exclude-libs,ALL |
Two copies | Two copies | Crashes; not usable |
--exclude-libs,libstdc++.a |
Two copies | One copy | Runs, but creates the two-copy state the reviewer was concerned about |
The reviewer's concern ("two copies of the same library will conflict") is valid. However, measurement shows the current structure already uses a single libstdc++, and --exclude-libs creates that concern rather than resolving it. Therefore the current code is left unchanged.
|
Sorry for the slow review! We appreciate the contribution. |
This PR was reopened (likely due to being reverted), so your approval was removed. Please request another review.
|
@BicMak - Would you mind merging main into the branch? There are a couple required checks that seem to have an issue running. |
|
@rascani Thank you for approving my PR. I've merged the latest main into the branch as requested. Could you please check it again? |
Problem
setup.shbuilds against the glibc 2.40 sysroot bundled with the Arm toolchain (14.3.rel1). As a result,llama_mainandlibllama_runner.sorequire GLIBC_2.38, GLIBCXX_3.4.32 and CXXABI_1.3.15, so onBookworm (glibc 2.36) the loader fails with
version 'GLIBC_2.38' not found.Changes
--os <bookworm|trixie>: build against the target Debian release's glibc. An arm64 sysroot isdownloaded with mmdebstrap, and libstdc++/libgcc are linked statically.
glibc_compat.c: the toolchain's static libstdc++ references__isoc23_strtoul, which glibc < 2.38does not provide, so linking fails. glibc 2.38 added this symbol for calls to
strtoulcompiled inC23 mode (it also accepts a
0bprefix). The behavior is the same for the inputs libstdc++ passes, sothe wrapper simply forwards to
strtoul. It is compiled and linked only when the sysroot's glibc isolder than 2.38. Aliasing it with
-Wl,--defsymdoes not work becausestrtoulis a shared-librarysymbol (verified).
--osthe behavior is unchanged (the toolchain's bundled glibc), and a warning points to--os.Scope and limits
toolchain (e.g. libgcc's unwinder) require glibc >= 2.35.
examples/raspberry_pi.pi4target is not tested.Test plan
Build on the host:
Run on a Raspberry Pi 5 (put
llama_mainandlibllama_runner.soin the same folder):--os bookworm--os bookworm--os trixieNote: the same Raspberry Pi 5 was used for every run. Bookworm and Trixie were installed on two separate
microSD cards, and the card was swapped to boot each OS.
"Pass" means
--helpprints its usage, and inference exits 0 and generates 26 tokens that contain "Paris"Sample output (Bookworm build, Bookworm OS)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
cc @nil-is-all @psiddh @AdrianLundell @digantdesai @rascani