Repository navigation
gh-160924: Don't run the build Python with RUNSHARED when cross-compiling - #161321
Conversation
…-compiling In a cross build the build Python is the --with-build-python interpreter. Running it with RUNSHARED puts the target's build directory on the library path, so a shared build Python of the same version loads the target's libpython, which fails when the target uses another libc. Add RUNSHARED_FOR_BUILD, empty when cross-compiling, and use it for $(PYTHON_FOR_BUILD). RUNSHARED itself is unchanged, so the PGO profile run (pythongh-141926) and TESTPYTHON still get it.
|
All commit authors signed the Contributor License Agreement. |
itamaro
left a comment
There was a problem hiding this comment.
thanks for the fix, I agree this is worth fixing.
I believe the compileall calls also need fixing (see inline comments).
|
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
|
Thanks for the review! You're right about I have made the requested changes; please review again |
|
Thanks for making the requested changes! @itamaro: please review the changes made to this pull request. |
|
Sorry, @aozmekik and @itamaro, I could not cleanly backport this to Please backport manually with cherry_picker, see the devguide for more information. |
|
Manual 3.15 backport: #161659 |
|
GH-161659 is a backport of this pull request to the 3.15 branch. |
When cross-compiling, the build Python is the
--with-build-pythoninterpreter, but the Makefile still ran it withRUNSHARED. With--enable-sharedthat puts the target's build directory onLD_LIBRARY_PATH, so a shared build Python of the same version loads the target'slibpython3.X.so.1.0instead of its own. glibc to glibc gets away with it; for a musl target the loader then fails on musl'slibc.so.This adds
RUNSHARED_FOR_BUILD, which is$(RUNSHARED)in native builds and empty in cross builds, and uses it where the Makefile runs$(PYTHON_FOR_BUILD).RUNSHAREDitself doesn't change, so the PGO profile run from gh-141926 still gets it.TESTPYTHONkeeps it too, since withHOSTRUNNERregrtest starts the target interpreter from it.Tested on Ubuntu 24.04 with a shared build Python from the same commit:
musl-gcc,--enable-shared): main fails atpybuilddir.txtwitherror while loading shared libraries: /lib/x86_64-linux-gnu/libc.so: invalid ELF header. With this change the build,make install(ensurepip) and the cross-built interpreter all work.--enable-optimizations: passes before and after, and the profile run still getsLD_LIBRARY_PATH. ClearingRUNSHAREDfor cross builds again, as before 3.15, breaks it (libpython3.16.so.1.0: cannot open shared object file).