On 2019/10/20 8:37, Branko Čibej wrote:
> On 20.10.2019 01:10, Yasuhito FUTATSUKI wrote:
>> On 2019/10/20 6:59, Branko Čibej wrote:
>>> On 19.10.2019 23:06, Branko Čibej wrote:
>>>> On 19.10.2019 19:55, Branko Čibej wrote:
>>>>> On 19.10.2019 11:45, Yasuhito FUTATSUKI wrote:
>>>>>> On 2019/10/18 8:39, Branko Čibej wrote:
>>>>>>> On 17.10.2019 23:46, Branko Čibej wrote:
>>>>>>>> On 17.10.2019 23:01, Daniel Shahaf wrote:
>>>>>>>>> Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
>>>>>>>>>> On 17.10.2019 22:37, Daniel Shahaf wrote:
>>>>>>>>>>> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>>>>>>>>>>>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>>>>>>>>>>>> Which reminds me: it'd be nice to set up a buildbot to run the
>>>>>>>>>>>>> build and
>>>>>>>>>>>>> test process under Python 3. Any committer can edit the
>>>>>>>>>>>>> buildbot
>>>>>>>>>>>>> scripts[1], but the question is which of the buildbot slaves
>>>>>>>>>>>>> has Python 3
>>>>>>>>>>>>> installed?
>>>>>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>>>>>>>>>>>
>>>>>>>>>>>> Been there, done that, bought the DVD.
>>>>>>>>>>> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing
>>>>>>>>>>> on that
>>>>>>>>>>> bot? They (currently) fail under python3.
>>>>>>>>>> I don't know. It's possible, I suppose, that the activation of
>>>>>>>>>> the
>>>>>>>>>> python3 virtual environment has no effect on the test driver. It
>>>>>>>>>> might
>>>>>>>>>> not be a bad idea to have the tests print the Python version
>>>>>>>>>> in the
>>>>>>>>>> summary and possibly in the log of every test case.
>>>>>>>>> What's the value of $(PYTHON) in Makefile? That's what the
>>>>>>>>> «check»
>>>>>>>>> target uses.
>>>>>>>> Yep, apparently that's the bug ... I'm testing script changes now,
>>>>>>>> along
>>>>>>>> with r1868566 for good measure.
>>>>>>> I've set up this:
>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig
>>>>>>>
>>>>>>> It will build and test the core libraries and swig-py bindings with
>>>>>>> Python 3 on the swig-py3 branch.
>>>>>>>
>>>>>>> I hope ... build #0 running as we speak.
>>>>>> Unfortunately build #2, which ran after upgrading SWIG and Python,
>>>>>> failed
>>>>>> to build SWIG Python bindings because of SWIG 4.0, as reported on
>>>>>> SVN-4818.
>>>>>>
>>>>>> This also affects on trunk with SWIG 4.0.
>>>>>> (e.g. https://ci.apache.org/builders/svn-x64-macosx-full/builds/2418)
>>>>>>
>>>>>> With attached patch on trunk (trunk_build_with_swig4_patch.txt) and
>>>>>> on swig-py3 (swig_py3_build_with_swig4_patch.txt), *.so can be
>>>>>> produced,
>>>>>> but the modules don't work correctly.
>>>>>>
>>>>>> It seems they were caused by incompatibility of Python code for proxy
>>>>>> object generated by SWIG, and it can not be resolved so simple....
>>>>>> (importlib module vs simply use 'import', absense of _swig_setattr()
>>>>>> and _swig_getattr(), etc.)
>>>>> My bad. I downgraded to Swig 3.0.12 and restarted the failed builds.
>>>>
>>>> Hm, that did not really help. On the swig-py3 branch:
>>>>
>>>> /bin/sh: line 1: 62481 Segmentation fault: 11
>>>> /srv/virtualenv-3/bin/python
>>>> /srv/buildbot/slave/svn-x64-macosx-local-python3-swig/build/subversion/bindings/swig/python/tests/run_all.py
>>>> make: *** [check-swig-py] Error 139
>>>>
>>>>
>>>> So, the Python bindings seem to have crashed the Python 3
>>>> interpreter on
>>>> macOS?
>>>
>>> Yes, they did:
>>>
>>> Process 28278 stopped
>>> * thread #2, queue = 'com.apple.main-thread', stop reason =
>>> EXC_BAD_ACCESS (code=1, address=0x48)
>>> frame #0: 0x00007fff373ab746 Python`PyErr_Restore + 61
>>> Python`PyErr_Restore:
>>> -> 0x7fff373ab746 <+61>: movq 0x48(%rbx), %rdi
>>> 0x7fff373ab74a <+65>: movq 0x50(%rbx), %r13
>>> 0x7fff373ab74e <+69>: movq 0x58(%rbx), %r12
>>> 0x7fff373ab752 <+73>: movq %r15, 0x48(%rbx)
>>> Target 0: (Python) stopped.
>>>
>>> frame #3: 0x0000000103800fea _core.so`PyInit__core at core.c:40158:12
>>> 40155 m = Py_InitModule((char *) SWIG_name, SwigMethods);
>>> 40156 #endif
>>> 40157
>>> -> 40158 md = d = PyModule_GetDict(m);
>>> 40159 (void)md;
>>> 40160
>>> 40161 SWIG_InitializeModule(0);
>>> (lldb) print m
>>> (PyObject *) $0 = 0x000000010279ad70
>>> (lldb) print *m
>>> (PyObject) $1 = {
>>> ob_refcnt = 785
>>> ob_type = 0x000000010026ea30
>>> }
>>
>> I guess the _core module linked incompatible Python library to execute.
>>
>> on
>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig/builds/6/steps/Build/logs/stdio
>> :
>> [[[
>> checking for compiling Python extensions... clang
>> checking for linking Python extensions... clang -bundle -undefined
>> dynamic_lookup -isysroot
>> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
>> -F/usr/local/opt/python/Frameworks -framework Python
>> checking for linking Python libraries... -bundle -undefined
>> dynamic_lookup -F/usr/local/opt/python/Frameworks -framework Python
>> ]]]
>
>
> Good catch. This certainly looks correct, it's where Homebrew installs
> python. However ...
>
> $ otool -L /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so
> /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so:
> /System/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.10)
> ...
>
>
> That is wrong. Looks like we'll have to set DYLD_LIBRARY_PATH so that it
> finds the Python 3 installation.
... or have to use Run-Path Dependent Libraries feature on link time,
if DYLD_LIBRARY_PATH cannot work well due to SIP.
https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/RunpathDependentLibraries.html
The link option in log above comes from result of
'build/get-py-info.py --link', so should we tweak build/get-py-info.py,
or add option to configure to pass Python dynamic library path?
Cheers,
--
Yasuhito FUTATSUKI <futatuki_at_poem.co.jp>
Received on 2019-10-20 06:36:10 CEST