On 2019/10/20 21:58, Branko Čibej wrote:
> On 20.10.2019 06:35, Yasuhito FUTATSUKI wrote:
>> On 2019/10/20 8:37, Branko Čibej wrote:
>>> On 20.10.2019 01:10, Yasuhito FUTATSUKI wrote:
>>>> On 2019/10/20 6:59, Branko Čibej wrote:
>>>>> On 19.10.2019 23:06, Branko Čibej wrote:
>>>>>> On 19.10.2019 19:55, Branko Čibej wrote:
>>>>>>> On 19.10.2019 11:45, Yasuhito FUTATSUKI wrote:
>>>>>>>> On 2019/10/18 8:39, Branko Čibej wrote:
>>>>>>>>> On 17.10.2019 23:46, Branko Čibej wrote:
>>>>>>>>>> On 17.10.2019 23:01, Daniel Shahaf wrote:
>>>>>>>>>>> Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
>>>>>>>>>>>> On 17.10.2019 22:37, Daniel Shahaf wrote:
>>>>>>>>>>>>> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>>>>>>>>>>>>>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>>>>>>>>>>>>>> Which reminds me: it'd be nice to set up a buildbot to
>>>>>>>>>>>>>>> run the
>>>>>>>>>>>>>>> build and
>>>>>>>>>>>>>>> test process under Python 3. Any committer can edit the
>>>>>>>>>>>>>>> buildbot
>>>>>>>>>>>>>>> scripts[1], but the question is which of the buildbot slaves
>>>>>>>>>>>>>>> has Python 3
>>>>>>>>>>>>>>> installed?
>>>>>>>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Been there, done that, bought the DVD.
>>>>>>>>>>>>> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing
>>>>>>>>>>>>> on that
>>>>>>>>>>>>> bot? They (currently) fail under python3.
>>>>>>>>>>>> I don't know. It's possible, I suppose, that the activation of
>>>>>>>>>>>> the
>>>>>>>>>>>> python3 virtual environment has no effect on the test
>>>>>>>>>>>> driver. It
>>>>>>>>>>>> might
>>>>>>>>>>>> not be a bad idea to have the tests print the Python version
>>>>>>>>>>>> in the
>>>>>>>>>>>> summary and possibly in the log of every test case.
>>>>>>>>>>> What's the value of $(PYTHON) in Makefile? That's what the
>>>>>>>>>>> «check»
>>>>>>>>>>> target uses.
>>>>>>>>>> Yep, apparently that's the bug ... I'm testing script changes
>>>>>>>>>> now,
>>>>>>>>>> along
>>>>>>>>>> with r1868566 for good measure.
>>>>>>>>> I've set up this:
>>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig
>>>>>>>>>
>>>>>>>>> It will build and test the core libraries and swig-py bindings
>>>>>>>>> with
>>>>>>>>> Python 3 on the swig-py3 branch.
>>>>>>>>>
>>>>>>>>> I hope ... build #0 running as we speak.
>>>>>>>> Unfortunately build #2, which ran after upgrading SWIG and Python,
>>>>>>>> failed
>>>>>>>> to build SWIG Python bindings because of SWIG 4.0, as reported on
>>>>>>>> SVN-4818.
>>>>>>>>
>>>>>>>> This also affects on trunk with SWIG 4.0.
>>>>>>>> (e.g.
>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-full/builds/2418)
>>>>>>>>
>>>>>>>> With attached patch on trunk (trunk_build_with_swig4_patch.txt) and
>>>>>>>> on swig-py3 (swig_py3_build_with_swig4_patch.txt), *.so can be
>>>>>>>> produced,
>>>>>>>> but the modules don't work correctly.
>>>>>>>>
>>>>>>>> It seems they were caused by incompatibility of Python code for
>>>>>>>> proxy
>>>>>>>> object generated by SWIG, and it can not be resolved so simple....
>>>>>>>> (importlib module vs simply use 'import', absense of
>>>>>>>> _swig_setattr()
>>>>>>>> and _swig_getattr(), etc.)
>>>>>>> My bad. I downgraded to Swig 3.0.12 and restarted the failed builds.
>>>>>>
>>>>>> Hm, that did not really help. On the swig-py3 branch:
>>>>>>
>>>>>> /bin/sh: line 1: 62481 Segmentation fault: 11
>>>>>> /srv/virtualenv-3/bin/python
>>>>>> /srv/buildbot/slave/svn-x64-macosx-local-python3-swig/build/subversion/bindings/swig/python/tests/run_all.py
>>>>>>
>>>>>> make: *** [check-swig-py] Error 139
>>>>>>
>>>>>>
>>>>>> So, the Python bindings seem to have crashed the Python 3
>>>>>> interpreter on
>>>>>> macOS?
>>>>>
>>>>> Yes, they did:
>>>>>
>>>>> Process 28278 stopped
>>>>> * thread #2, queue = 'com.apple.main-thread', stop reason =
>>>>> EXC_BAD_ACCESS (code=1, address=0x48)
>>>>> frame #0: 0x00007fff373ab746 Python`PyErr_Restore + 61
>>>>> Python`PyErr_Restore:
>>>>> -> 0x7fff373ab746 <+61>: movq 0x48(%rbx), %rdi
>>>>> 0x7fff373ab74a <+65>: movq 0x50(%rbx), %r13
>>>>> 0x7fff373ab74e <+69>: movq 0x58(%rbx), %r12
>>>>> 0x7fff373ab752 <+73>: movq %r15, 0x48(%rbx)
>>>>> Target 0: (Python) stopped.
>>>>>
>>>>> frame #3: 0x0000000103800fea _core.so`PyInit__core at core.c:40158:12
>>>>> 40155 m = Py_InitModule((char *) SWIG_name, SwigMethods);
>>>>> 40156 #endif
>>>>> 40157
>>>>> -> 40158 md = d = PyModule_GetDict(m);
>>>>> 40159 (void)md;
>>>>> 40160
>>>>> 40161 SWIG_InitializeModule(0);
>>>>> (lldb) print m
>>>>> (PyObject *) $0 = 0x000000010279ad70
>>>>> (lldb) print *m
>>>>> (PyObject) $1 = {
>>>>> ob_refcnt = 785
>>>>> ob_type = 0x000000010026ea30
>>>>> }
>>>>
>>>> I guess the _core module linked incompatible Python library to execute.
>>>>
>>>> on
>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig/builds/6/steps/Build/logs/stdio
>>>>
>>>> :
>>>> [[[
>>>> checking for compiling Python extensions... clang
>>>> checking for linking Python extensions... clang -bundle -undefined
>>>> dynamic_lookup -isysroot
>>>> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
>>>>
>>>> -F/usr/local/opt/python/Frameworks -framework Python
>>>> checking for linking Python libraries... -bundle -undefined
>>>> dynamic_lookup -F/usr/local/opt/python/Frameworks -framework Python
>>>> ]]]
>>>
>>>
>>> Good catch. This certainly looks correct, it's where Homebrew installs
>>> python. However ...
>>>
>>> $ otool -L /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so
>>> /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so:
>>> /System/Library/Frameworks/Python.framework/Versions/2.7/Python
>>> (compatibility version 2.7.0, current version 2.7.10)
>>> ...
>>>
>>>
>>> That is wrong. Looks like we'll have to set DYLD_LIBRARY_PATH so that it
>>> finds the Python 3 installation.
>
> (Actually that would be DYLD_FRAMEWORK_PATH.)
>
>> ... or have to use Run-Path Dependent Libraries feature on link time,
>> if DYLD_LIBRARY_PATH cannot work well due to SIP.
>>
>> https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/RunpathDependentLibraries.html
>>
>>
>> The link option in log above comes from result of
>> 'build/get-py-info.py --link', so should we tweak build/get-py-info.py,
>> or add option to configure to pass Python dynamic library path?
>
>
> Using RPATH for the Python framework on macOS would be the right
> solution, but it doesn't solve the problem. It looks like we need some
> magic incantation on macOS to explicitly pick the Python 3 framework;
> using just the -F option doesn't work as expected, the path to the
> system default Python 2.7 framework is hard-coded in to the extension
> libraries.
Umm.... it seems extension libraries actually don't need to link
"Python" dynamic library.
e.g.
$ otool -L `python -c 'import _socket;print(_socket.__file__)'`
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_socket.so:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
_core.so built on FreeBSD:
$ ldd subversion/bindings/swig/python/.libs/_core.so
subversion/bindings/swig/python/.libs/_core.so:
libsvn_swig_py-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py/.libs/libsvn_swig_py-1.so.0 (0x801298000)
libsvn_client-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_client/.libs/libsvn_client-1.so.0 (0x8014ac000)
libsvn_wc-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_wc/.libs/libsvn_wc-1.so.0 (0x801730000)
libsvn_diff-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_diff/.libs/libsvn_diff-1.so.0 (0x8019d5000)
libsvn_ra-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_ra/.libs/libsvn_ra-1.so.0 (0x801bec000)
libsvn_ra_local-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_ra_local/.libs/libsvn_ra_local-1.so.0 (0x801df8000)
libsvn_repos-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_repos/.libs/libsvn_repos-1.so.0 (0x802001000)
libsvn_fs-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_fs/.libs/libsvn_fs-1.so.0 (0x80223c000)
libsvn_fs_fs-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_fs_fs/.libs/libsvn_fs_fs-1.so.0 (0x802449000)
libsvn_fs_x-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_fs_x/.libs/libsvn_fs_x-1.so.0 (0x802698000)
libsvn_fs_util-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_fs_util/.libs/libsvn_fs_util-1.so.0 (0x8028e7000)
libsvn_ra_svn-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_ra_svn/.libs/libsvn_ra_svn-1.so.0 (0x802ae9000)
libsvn_ra_serf-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_ra_serf/.libs/libsvn_ra_serf-1.so.0 (0x802d0d000)
libserf-1.so.1 => /usr/local/lib/libserf-1.so.1 (0x802f3e000)
libsvn_delta-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_delta/.libs/libsvn_delta-1.so.0 (0x803159000)
libsvn_subr-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_subr/.libs/libsvn_subr-1.so.0 (0x803377000)
libaprutil-1.so.0 => /usr/local/lib/libaprutil-1.so.0 (0x803608000)
libexpat.so.1 => /usr/local/lib/libexpat.so.1 (0x803835000)
libz.so.6 => /lib/libz.so.6 (0x803a5f000)
libsqlite3.so.0 => /usr/local/lib/libsqlite3.so.0 (0x803c77000)
libmagic.so.4 => /usr/lib/libmagic.so.4 (0x803fd9000)
liblz4.so.1 => /usr/local/lib/liblz4.so.1 (0x8041fc000)
libutf8proc.so.2 => /usr/local/lib/libutf8proc.so.2 (0x80442b000)
libapr-1.so.0 => /usr/local/lib/libapr-1.so.0 (0x804675000)
libintl.so.8 => /usr/local/lib/libintl.so.8 (0x8048af000)
libthr.so.3 => /lib/libthr.so.3 (0x804aba000)
libc.so.7 => /lib/libc.so.7 (0x800823000)
libssl.so.8 => /usr/lib/libssl.so.8 (0x804ce2000)
libcrypto.so.8 => /lib/libcrypto.so.8 (0x805000000)
libcrypt.so.5 => /lib/libcrypt.so.5 (0x80546f000)
libdb-5.3.so.0 => /usr/local/lib/libdb-5.3.so.0 (0x80568e000)
libgdbm.so.6 => /usr/local/lib/libgdbm.so.6 (0x805a33000)
--
Yasuhito FUTATSUKI <futatuki_at_poem.co.jp>
Received on 2019-10-21 01:15:45 CEST