[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: SWIG Python bindings build with SWIG 4.0

From: Branko Čibej <brane_at_apache.org>
Date: Sun, 20 Oct 2019 01:37:23 +0200

On 20.10.2019 01:10, Yasuhito FUTATSUKI wrote:
> On 2019/10/20 6:59, Branko Čibej wrote:
>> On 19.10.2019 23:06, Branko Čibej wrote:
>>> On 19.10.2019 19:55, Branko Čibej wrote:
>>>> On 19.10.2019 11:45, Yasuhito FUTATSUKI wrote:
>>>>> On 2019/10/18 8:39, Branko Čibej wrote:
>>>>>> On 17.10.2019 23:46, Branko Čibej wrote:
>>>>>>> On 17.10.2019 23:01, Daniel Shahaf wrote:
>>>>>>>> Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
>>>>>>>>> On 17.10.2019 22:37, Daniel Shahaf wrote:
>>>>>>>>>> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>>>>>>>>>>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>>>>>>>>>>> Which reminds me: it'd be nice to set up a buildbot to run the
>>>>>>>>>>>> build and
>>>>>>>>>>>> test process under Python 3. Any committer can edit the
>>>>>>>>>>>> buildbot
>>>>>>>>>>>> scripts[1], but the question is which of the buildbot slaves
>>>>>>>>>>>> has Python 3
>>>>>>>>>>>> installed?
>>>>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>>>>>>>>>>
>>>>>>>>>>> Been there, done that, bought the DVD.
>>>>>>>>>> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing
>>>>>>>>>> on that
>>>>>>>>>> bot?  They (currently) fail under python3.
>>>>>>>>> I don't know. It's possible, I suppose, that the activation of
>>>>>>>>> the
>>>>>>>>> python3 virtual environment has no effect on the test driver. It
>>>>>>>>> might
>>>>>>>>> not be a bad idea to have the tests print the Python version
>>>>>>>>> in the
>>>>>>>>> summary and possibly in the log of every test case.
>>>>>>>> What's the value of $(PYTHON) in Makefile?  That's what the
>>>>>>>> «check»
>>>>>>>> target uses.
>>>>>>> Yep, apparently that's the bug ... I'm testing script changes now,
>>>>>>> along
>>>>>>> with r1868566 for good measure.
>>>>>> I've set up this:
>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig
>>>>>>
>>>>>> It will build and test the core libraries and swig-py bindings with
>>>>>> Python 3 on the swig-py3 branch.
>>>>>>
>>>>>> I hope ... build #0 running as we speak.
>>>>> Unfortunately build #2, which ran after upgrading SWIG and Python,
>>>>> failed
>>>>> to build SWIG Python bindings because of SWIG 4.0, as reported on
>>>>> SVN-4818.
>>>>>
>>>>> This also affects on trunk with SWIG 4.0.
>>>>> (e.g. https://ci.apache.org/builders/svn-x64-macosx-full/builds/2418)
>>>>>
>>>>> With attached patch on trunk (trunk_build_with_swig4_patch.txt) and
>>>>> on swig-py3 (swig_py3_build_with_swig4_patch.txt), *.so can be
>>>>> produced,
>>>>> but the modules don't work correctly.
>>>>>
>>>>> It seems they were caused by incompatibility of Python code for proxy
>>>>> object generated by SWIG, and it can not be resolved so simple....
>>>>> (importlib module vs simply use 'import', absense of _swig_setattr()
>>>>> and _swig_getattr(), etc.)
>>>> My bad. I downgraded to Swig 3.0.12 and restarted the failed builds.
>>>
>>> Hm, that did not really help. On the swig-py3 branch:
>>>
>>> /bin/sh: line 1: 62481 Segmentation fault: 11 
>>> /srv/virtualenv-3/bin/python
>>> /srv/buildbot/slave/svn-x64-macosx-local-python3-swig/build/subversion/bindings/swig/python/tests/run_all.py
>>> make: *** [check-swig-py] Error 139
>>>
>>>
>>> So, the Python bindings seem to have crashed the Python 3
>>> interpreter on
>>> macOS?
>>
>> Yes, they did:
>>
>> Process 28278 stopped
>> * thread #2, queue = 'com.apple.main-thread', stop reason =
>> EXC_BAD_ACCESS (code=1, address=0x48)
>>      frame #0: 0x00007fff373ab746 Python`PyErr_Restore + 61
>> Python`PyErr_Restore:
>> ->  0x7fff373ab746 <+61>: movq   0x48(%rbx), %rdi
>>      0x7fff373ab74a <+65>: movq   0x50(%rbx), %r13
>>      0x7fff373ab74e <+69>: movq   0x58(%rbx), %r12
>>      0x7fff373ab752 <+73>: movq   %r15, 0x48(%rbx)
>> Target 0: (Python) stopped.
>>
>> frame #3: 0x0000000103800fea _core.so`PyInit__core at core.c:40158:12
>>     40155      m = Py_InitModule((char *) SWIG_name, SwigMethods);
>>     40156    #endif
>>     40157   
>> -> 40158      md = d = PyModule_GetDict(m);
>>     40159      (void)md;
>>     40160   
>>     40161      SWIG_InitializeModule(0);
>> (lldb) print m
>> (PyObject *) $0 = 0x000000010279ad70
>> (lldb) print *m
>> (PyObject) $1 = {
>>    ob_refcnt = 785
>>    ob_type = 0x000000010026ea30
>> }
>
> I guess the _core module linked incompatible Python library to execute.
>
> on
> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig/builds/6/steps/Build/logs/stdio
> :
> [[[
> checking for compiling Python extensions... clang
> checking for linking Python extensions... clang -bundle -undefined
> dynamic_lookup -isysroot
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
> -F/usr/local/opt/python/Frameworks -framework Python
> checking for linking Python libraries... -bundle -undefined
> dynamic_lookup -F/usr/local/opt/python/Frameworks -framework Python
> ]]]

Good catch. This certainly looks correct, it's where Homebrew installs
python. However ...

$ otool -L /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so
/opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so:
        /System/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.10)
        ...

That is wrong. Looks like we'll have to set DYLD_LIBRARY_PATH so that it
finds the Python 3 installation.

-- Brane
Received on 2019-10-20 01:37:25 CEST

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.