On 20.10.2019 06:35, Yasuhito FUTATSUKI wrote:
> On 2019/10/20 8:37, Branko Čibej wrote:
>> On 20.10.2019 01:10, Yasuhito FUTATSUKI wrote:
>>> On 2019/10/20 6:59, Branko Čibej wrote:
>>>> On 19.10.2019 23:06, Branko Čibej wrote:
>>>>> On 19.10.2019 19:55, Branko Čibej wrote:
>>>>>> On 19.10.2019 11:45, Yasuhito FUTATSUKI wrote:
>>>>>>> On 2019/10/18 8:39, Branko Čibej wrote:
>>>>>>>> On 17.10.2019 23:46, Branko Čibej wrote:
>>>>>>>>> On 17.10.2019 23:01, Daniel Shahaf wrote:
>>>>>>>>>> Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
>>>>>>>>>>> On 17.10.2019 22:37, Daniel Shahaf wrote:
>>>>>>>>>>>> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>>>>>>>>>>>>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>>>>>>>>>>>>> Which reminds me: it'd be nice to set up a buildbot to
>>>>>>>>>>>>>> run the
>>>>>>>>>>>>>> build and
>>>>>>>>>>>>>> test process under Python 3. Any committer can edit the
>>>>>>>>>>>>>> buildbot
>>>>>>>>>>>>>> scripts[1], but the question is which of the buildbot slaves
>>>>>>>>>>>>>> has Python 3
>>>>>>>>>>>>>> installed?
>>>>>>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>>>>>>>>>>>>
>>>>>>>>>>>>> Been there, done that, bought the DVD.
>>>>>>>>>>>> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing
>>>>>>>>>>>> on that
>>>>>>>>>>>> bot? They (currently) fail under python3.
>>>>>>>>>>> I don't know. It's possible, I suppose, that the activation of
>>>>>>>>>>> the
>>>>>>>>>>> python3 virtual environment has no effect on the test
>>>>>>>>>>> driver. It
>>>>>>>>>>> might
>>>>>>>>>>> not be a bad idea to have the tests print the Python version
>>>>>>>>>>> in the
>>>>>>>>>>> summary and possibly in the log of every test case.
>>>>>>>>>> What's the value of $(PYTHON) in Makefile? That's what the
>>>>>>>>>> «check»
>>>>>>>>>> target uses.
>>>>>>>>> Yep, apparently that's the bug ... I'm testing script changes
>>>>>>>>> now,
>>>>>>>>> along
>>>>>>>>> with r1868566 for good measure.
>>>>>>>> I've set up this:
>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig
>>>>>>>>
>>>>>>>> It will build and test the core libraries and swig-py bindings
>>>>>>>> with
>>>>>>>> Python 3 on the swig-py3 branch.
>>>>>>>>
>>>>>>>> I hope ... build #0 running as we speak.
>>>>>>> Unfortunately build #2, which ran after upgrading SWIG and Python,
>>>>>>> failed
>>>>>>> to build SWIG Python bindings because of SWIG 4.0, as reported on
>>>>>>> SVN-4818.
>>>>>>>
>>>>>>> This also affects on trunk with SWIG 4.0.
>>>>>>> (e.g.
>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-full/builds/2418)
>>>>>>>
>>>>>>> With attached patch on trunk (trunk_build_with_swig4_patch.txt) and
>>>>>>> on swig-py3 (swig_py3_build_with_swig4_patch.txt), *.so can be
>>>>>>> produced,
>>>>>>> but the modules don't work correctly.
>>>>>>>
>>>>>>> It seems they were caused by incompatibility of Python code for
>>>>>>> proxy
>>>>>>> object generated by SWIG, and it can not be resolved so simple....
>>>>>>> (importlib module vs simply use 'import', absense of
>>>>>>> _swig_setattr()
>>>>>>> and _swig_getattr(), etc.)
>>>>>> My bad. I downgraded to Swig 3.0.12 and restarted the failed builds.
>>>>>
>>>>> Hm, that did not really help. On the swig-py3 branch:
>>>>>
>>>>> /bin/sh: line 1: 62481 Segmentation fault: 11
>>>>> /srv/virtualenv-3/bin/python
>>>>> /srv/buildbot/slave/svn-x64-macosx-local-python3-swig/build/subversion/bindings/swig/python/tests/run_all.py
>>>>>
>>>>> make: *** [check-swig-py] Error 139
>>>>>
>>>>>
>>>>> So, the Python bindings seem to have crashed the Python 3
>>>>> interpreter on
>>>>> macOS?
>>>>
>>>> Yes, they did:
>>>>
>>>> Process 28278 stopped
>>>> * thread #2, queue = 'com.apple.main-thread', stop reason =
>>>> EXC_BAD_ACCESS (code=1, address=0x48)
>>>> frame #0: 0x00007fff373ab746 Python`PyErr_Restore + 61
>>>> Python`PyErr_Restore:
>>>> -> 0x7fff373ab746 <+61>: movq 0x48(%rbx), %rdi
>>>> 0x7fff373ab74a <+65>: movq 0x50(%rbx), %r13
>>>> 0x7fff373ab74e <+69>: movq 0x58(%rbx), %r12
>>>> 0x7fff373ab752 <+73>: movq %r15, 0x48(%rbx)
>>>> Target 0: (Python) stopped.
>>>>
>>>> frame #3: 0x0000000103800fea _core.so`PyInit__core at core.c:40158:12
>>>> 40155 m = Py_InitModule((char *) SWIG_name, SwigMethods);
>>>> 40156 #endif
>>>> 40157
>>>> -> 40158 md = d = PyModule_GetDict(m);
>>>> 40159 (void)md;
>>>> 40160
>>>> 40161 SWIG_InitializeModule(0);
>>>> (lldb) print m
>>>> (PyObject *) $0 = 0x000000010279ad70
>>>> (lldb) print *m
>>>> (PyObject) $1 = {
>>>> ob_refcnt = 785
>>>> ob_type = 0x000000010026ea30
>>>> }
>>>
>>> I guess the _core module linked incompatible Python library to execute.
>>>
>>> on
>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig/builds/6/steps/Build/logs/stdio
>>>
>>> :
>>> [[[
>>> checking for compiling Python extensions... clang
>>> checking for linking Python extensions... clang -bundle -undefined
>>> dynamic_lookup -isysroot
>>> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
>>>
>>> -F/usr/local/opt/python/Frameworks -framework Python
>>> checking for linking Python libraries... -bundle -undefined
>>> dynamic_lookup -F/usr/local/opt/python/Frameworks -framework Python
>>> ]]]
>>
>>
>> Good catch. This certainly looks correct, it's where Homebrew installs
>> python. However ...
>>
>> $ otool -L /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so
>> /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so:
>> /System/Library/Frameworks/Python.framework/Versions/2.7/Python
>> (compatibility version 2.7.0, current version 2.7.10)
>> ...
>>
>>
>> That is wrong. Looks like we'll have to set DYLD_LIBRARY_PATH so that it
>> finds the Python 3 installation.
(Actually that would be DYLD_FRAMEWORK_PATH.)
> ... or have to use Run-Path Dependent Libraries feature on link time,
> if DYLD_LIBRARY_PATH cannot work well due to SIP.
>
> https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/RunpathDependentLibraries.html
>
>
> The link option in log above comes from result of
> 'build/get-py-info.py --link', so should we tweak build/get-py-info.py,
> or add option to configure to pass Python dynamic library path?
Using RPATH for the Python framework on macOS would be the right
solution, but it doesn't solve the problem. It looks like we need some
magic incantation on macOS to explicitly pick the Python 3 framework;
using just the -F option doesn't work as expected, the path to the
system default Python 2.7 framework is hard-coded in to the extension
libraries.
-- Brane
Received on 2019-10-20 14:58:24 CEST