On 21.10.2019 01:15, Yasuhito FUTATSUKI wrote:
> On 2019/10/20 21:58, Branko Čibej wrote:
>> On 20.10.2019 06:35, Yasuhito FUTATSUKI wrote:
>>> On 2019/10/20 8:37, Branko Čibej wrote:
>>>> On 20.10.2019 01:10, Yasuhito FUTATSUKI wrote:
>>>>> On 2019/10/20 6:59, Branko Čibej wrote:
>>>>>> On 19.10.2019 23:06, Branko Čibej wrote:
>>>>>>> On 19.10.2019 19:55, Branko Čibej wrote:
>>>>>>>> On 19.10.2019 11:45, Yasuhito FUTATSUKI wrote:
>>>>>>>>> On 2019/10/18 8:39, Branko Čibej wrote:
>>>>>>>>>> On 17.10.2019 23:46, Branko Čibej wrote:
>>>>>>>>>>> On 17.10.2019 23:01, Daniel Shahaf wrote:
>>>>>>>>>>>> Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
>>>>>>>>>>>>> On 17.10.2019 22:37, Daniel Shahaf wrote:
>>>>>>>>>>>>>> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>>>>>>>>>>>>>>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>>>>>>>>>>>>>>> Which reminds me: it'd be nice to set up a buildbot to
>>>>>>>>>>>>>>>> run the
>>>>>>>>>>>>>>>> build and
>>>>>>>>>>>>>>>> test process under Python 3. Any committer can edit the
>>>>>>>>>>>>>>>> buildbot
>>>>>>>>>>>>>>>> scripts[1], but the question is which of the buildbot
>>>>>>>>>>>>>>>> slaves
>>>>>>>>>>>>>>>> has Python 3
>>>>>>>>>>>>>>>> installed?
>>>>>>>>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Been there, done that, bought the DVD.
>>>>>>>>>>>>>> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing
>>>>>>>>>>>>>> on that
>>>>>>>>>>>>>> bot? They (currently) fail under python3.
>>>>>>>>>>>>> I don't know. It's possible, I suppose, that the
>>>>>>>>>>>>> activation of
>>>>>>>>>>>>> the
>>>>>>>>>>>>> python3 virtual environment has no effect on the test
>>>>>>>>>>>>> driver. It
>>>>>>>>>>>>> might
>>>>>>>>>>>>> not be a bad idea to have the tests print the Python version
>>>>>>>>>>>>> in the
>>>>>>>>>>>>> summary and possibly in the log of every test case.
>>>>>>>>>>>> What's the value of $(PYTHON) in Makefile? That's what the
>>>>>>>>>>>> «check»
>>>>>>>>>>>> target uses.
>>>>>>>>>>> Yep, apparently that's the bug ... I'm testing script changes
>>>>>>>>>>> now,
>>>>>>>>>>> along
>>>>>>>>>>> with r1868566 for good measure.
>>>>>>>>>> I've set up this:
>>>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig
>>>>>>>>>>
>>>>>>>>>> It will build and test the core libraries and swig-py bindings
>>>>>>>>>> with
>>>>>>>>>> Python 3 on the swig-py3 branch.
>>>>>>>>>>
>>>>>>>>>> I hope ... build #0 running as we speak.
>>>>>>>>> Unfortunately build #2, which ran after upgrading SWIG and
>>>>>>>>> Python,
>>>>>>>>> failed
>>>>>>>>> to build SWIG Python bindings because of SWIG 4.0, as reported on
>>>>>>>>> SVN-4818.
>>>>>>>>>
>>>>>>>>> This also affects on trunk with SWIG 4.0.
>>>>>>>>> (e.g.
>>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-full/builds/2418)
>>>>>>>>>
>>>>>>>>> With attached patch on trunk
>>>>>>>>> (trunk_build_with_swig4_patch.txt) and
>>>>>>>>> on swig-py3 (swig_py3_build_with_swig4_patch.txt), *.so can be
>>>>>>>>> produced,
>>>>>>>>> but the modules don't work correctly.
>>>>>>>>>
>>>>>>>>> It seems they were caused by incompatibility of Python code for
>>>>>>>>> proxy
>>>>>>>>> object generated by SWIG, and it can not be resolved so
>>>>>>>>> simple....
>>>>>>>>> (importlib module vs simply use 'import', absense of
>>>>>>>>> _swig_setattr()
>>>>>>>>> and _swig_getattr(), etc.)
>>>>>>>> My bad. I downgraded to Swig 3.0.12 and restarted the failed
>>>>>>>> builds.
>>>>>>>
>>>>>>> Hm, that did not really help. On the swig-py3 branch:
>>>>>>>
>>>>>>> /bin/sh: line 1: 62481 Segmentation fault: 11
>>>>>>> /srv/virtualenv-3/bin/python
>>>>>>> /srv/buildbot/slave/svn-x64-macosx-local-python3-swig/build/subversion/bindings/swig/python/tests/run_all.py
>>>>>>>
>>>>>>>
>>>>>>> make: *** [check-swig-py] Error 139
>>>>>>>
>>>>>>>
>>>>>>> So, the Python bindings seem to have crashed the Python 3
>>>>>>> interpreter on
>>>>>>> macOS?
>>>>>>
>>>>>> Yes, they did:
>>>>>>
>>>>>> Process 28278 stopped
>>>>>> * thread #2, queue = 'com.apple.main-thread', stop reason =
>>>>>> EXC_BAD_ACCESS (code=1, address=0x48)
>>>>>> frame #0: 0x00007fff373ab746 Python`PyErr_Restore + 61
>>>>>> Python`PyErr_Restore:
>>>>>> -> 0x7fff373ab746 <+61>: movq 0x48(%rbx), %rdi
>>>>>> 0x7fff373ab74a <+65>: movq 0x50(%rbx), %r13
>>>>>> 0x7fff373ab74e <+69>: movq 0x58(%rbx), %r12
>>>>>> 0x7fff373ab752 <+73>: movq %r15, 0x48(%rbx)
>>>>>> Target 0: (Python) stopped.
>>>>>>
>>>>>> frame #3: 0x0000000103800fea _core.so`PyInit__core at
>>>>>> core.c:40158:12
>>>>>> 40155 m = Py_InitModule((char *) SWIG_name, SwigMethods);
>>>>>> 40156 #endif
>>>>>> 40157
>>>>>> -> 40158 md = d = PyModule_GetDict(m);
>>>>>> 40159 (void)md;
>>>>>> 40160
>>>>>> 40161 SWIG_InitializeModule(0);
>>>>>> (lldb) print m
>>>>>> (PyObject *) $0 = 0x000000010279ad70
>>>>>> (lldb) print *m
>>>>>> (PyObject) $1 = {
>>>>>> ob_refcnt = 785
>>>>>> ob_type = 0x000000010026ea30
>>>>>> }
>>>>>
>>>>> I guess the _core module linked incompatible Python library to
>>>>> execute.
>>>>>
>>>>> on
>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig/builds/6/steps/Build/logs/stdio
>>>>>
>>>>>
>>>>> :
>>>>> [[[
>>>>> checking for compiling Python extensions... clang
>>>>> checking for linking Python extensions... clang -bundle -undefined
>>>>> dynamic_lookup -isysroot
>>>>> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
>>>>>
>>>>>
>>>>> -F/usr/local/opt/python/Frameworks -framework Python
>>>>> checking for linking Python libraries... -bundle -undefined
>>>>> dynamic_lookup -F/usr/local/opt/python/Frameworks -framework Python
>>>>> ]]]
>>>>
>>>>
>>>> Good catch. This certainly looks correct, it's where Homebrew installs
>>>> python. However ...
>>>>
>>>> $ otool -L /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so
>>>> /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so:
>>>> /System/Library/Frameworks/Python.framework/Versions/2.7/Python
>>>> (compatibility version 2.7.0, current version 2.7.10)
>>>> ...
>>>>
>>>>
>>>> That is wrong. Looks like we'll have to set DYLD_LIBRARY_PATH so
>>>> that it
>>>> finds the Python 3 installation.
>>
>> (Actually that would be DYLD_FRAMEWORK_PATH.)
>>
>>> ... or have to use Run-Path Dependent Libraries feature on link time,
>>> if DYLD_LIBRARY_PATH cannot work well due to SIP.
>>>
>>> https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/RunpathDependentLibraries.html
>>>
>>>
>>>
>>> The link option in log above comes from result of
>>> 'build/get-py-info.py --link', so should we tweak build/get-py-info.py,
>>> or add option to configure to pass Python dynamic library path?
>>
>>
>> Using RPATH for the Python framework on macOS would be the right
>> solution, but it doesn't solve the problem. It looks like we need some
>> magic incantation on macOS to explicitly pick the Python 3 framework;
>> using just the -F option doesn't work as expected, the path to the
>> system default Python 2.7 framework is hard-coded in to the extension
>> libraries.
>
> Umm.... it seems extension libraries actually don't need to link
> "Python" dynamic library.
>
> e.g.
> $ otool -L `python -c 'import _socket;print(_socket.__file__)'`
> /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_socket.so:
>
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0,
> current version 1252.250.1)
You're right. Fixed in r1868674, this makes the Python bindings tests
work on my laptop, with both the system Python 2 and the Homebrewed
Python 3. I've forced the tests to run on the buildslave.
-- Brane
Received on 2019-10-21 04:11:54 CEST