[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Python3 work [was: The run up to Subversion 1.13.0]

From: Johan Corveleyn <jcorvel_at_gmail.com>
Date: Sun, 29 Sep 2019 16:36:48 +0200

On Sun, Sep 29, 2019 at 2:00 PM Yasuhito FUTATSUKI <futatuki_at_poem.co.jp> wrote:
>
> On 2019/09/28 17:17, Yasuhito FUTATSUKI wrote:
> > On 2019/09/28 5:51, Johan Corveleyn wrote:
>
> >> [[[
> >> Testing Release configuration on local repository.
> >> -- Running Swig Python tests --
> >> ..........................................................C:\Python37\lib\subprocess.py:858:
> >> ResourceWarning: subprocess
> >> 8548 is still running
> >> ResourceWarning, source=self)
> >> ResourceWarning: Enable tracemalloc to get the object allocation traceback
> >> C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file
> >> <_io.BufferedReader name=3>
> >> testMethod()
> >> ResourceWarning: Enable tracemalloc to get the object allocation traceback
> >> .C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file
> >> <_io.BufferedReader name='R:\\temp\\tmpryeb61g1'>
> >> testMethod()
> >> ResourceWarning: Enable tracemalloc to get the object allocation traceback
> >> ..............................................................................................
> >> ----------------------------------------------------------------------
> >> Ran 153 tests in 66.182s
> >>
> >> OK
> >> ]]]
> >
> > On FreeBSD those warning are also produced.
> >
> > [[[
> > futatuki_at_retina-alpha[15] make check-swig-py
> > if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py
> > ........................................................../usr/local/lib/python3.7/subprocess.py:858: ResourceWarning: subprocess 81136 is still running
> > ResourceWarning, source=self)
> > ResourceWarning: Enable tracemalloc to get the object allocation traceback
> > /usr/local/lib/python3.7/unittest/case.py:615: ResourceWarning: unclosed file <_io.BufferedReader name=4>
> > testMethod()
> > ResourceWarning: Enable tracemalloc to get the object allocation traceback
> > ./usr/local/lib/python3.7/unittest/case.py:615: ResourceWarning: unclosed file <_io.BufferedReader name='/tmp/tmporh5038d'>
> > testMethod()
> > ResourceWarning: Enable tracemalloc to get the object allocation traceback
> > ..............................................................................................
> > ----------------------------------------------------------------------
> > Ran 153 tests in 10.773s
> >
> > OK
> > ]]]
> >
> > I think these warnings comes from difference of warning between Python 2
> > and Python 3, and not affect quality of swig Python binding itself but
> > affect quality of its test code.
>
> This was partly wrong. The warning caused by subprocess comes from
> class svn.fs.FileDiff.get_pipe(), not in test code but in support class.
>
> >
> > ResourceWarning itself appeared in Python 3.2
> >
> > https://docs.python.org/3/whatsnew/3.2.html
> > (section "Other Language Changes")
> >
> > If we can allow this warning and want to hide it, we can use
> > environment variable PYTHONWARNINGS='ignore::ResourceWarning::',
> > as the document above says.
> >
> > [[[
> > futatuki_at_retina-alpha[17] env PYTHONWARNINGS='ignore::ResourceWarning::' make check-swig-py
> > if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py
> > .........................................................................................................................................................
> > ----------------------------------------------------------------------
> > Ran 153 tests in 10.757s
> >
> > OK
> > ]]]
> >
> > Of course, we can also resolve those warnings by fixing clean up code
> > on tests.
>
> The patch attached is expected to fix these warnings. I already tested
> on FreeBSD with Python 3.7 (treat ResourceWarning as an error).
>
> [[[
> futatuki_at_retina-alpha[113] env PYTHONWARNINGS='error::ResourceWarning::' make check-swig-py
> if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py
> .........................................................................................................................................................
> ----------------------------------------------------------------------
> Ran 153 tests in 11.040s
>
> OK
> ]]]

On my Windows setup, there is still 1 ResourceWarning (before your
patch there were 3, now only 1 remains).

[[[
C:\research\svn\dev\swig-py3>runtest.bat --swig=python
'ruby' is not recognized as an internal or external command,
operable program or batch file.
Testing Release configuration on local repository.
-- Running Swig Python tests --
..........................................................C:\Python37\lib\subprocess.py:858:
ResourceWarning: subprocess
 1408 is still running
  ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
...............................................................................................
----------------------------------------------------------------------
Ran 153 tests in 63.597s

OK
]]]

When I treat ResourceWarning as an error, I get a lot more output,
especially before the test runs start. I suppose those are unrelated
to that one RW above.

[[[
C:\research\svn\dev\swig-py3>set PYTHONWARNINGS=error::ResourceWarning::

C:\research\svn\dev\swig-py3>runtest.bat --swig=python
Exception ignored in: <_io.FileIO name='build.conf' mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name='build.conf'
mode='r' encoding='cp1252'>
Exception ignored in: <_io.FileIO
name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr\\include\\apr_version.h'
 mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr\\include
\\apr_version.h' mode='r' encoding='cp1252'>
Exception ignored in: <_io.FileIO
name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\include\\apu_versi
on.h' mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\in
clude\\apu_version.h' mode='r' encoding='cp1252'>
Exception ignored in: <_io.FileIO
name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\xml/expat/lib\\exp
at.h' mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\xm
l/expat/lib\\expat.h' mode='r' encoding='cp1252'>
Exception ignored in: <_io.FileIO
name='C:\\research\\svn\\dev\\deps\\zlib-1.2.11\\zlib.h' mode='rb'
closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='C:\\research\\svn\\dev\\deps\\zlib-1.2.11\\zlib.h' mode='r' enco
ding='cp1252'>
Exception ignored in: <_io.FileIO
name='C:\\research\\svn\\dev\\deps\\sqlite-amalgamation-3.22.0.0\\sqlite3.h'
mode='rb'
 closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='C:\\research\\svn\\dev\\deps\\sqlite-amalgamation-3.22.0.0\\sqli
te3.h' mode='r' encoding='cp1252'>
Exception ignored in: <_io.FileIO
name='subversion\\libsvn_subr\\lz4\\lz4internal.h' mode='rb'
closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='subversion\\libsvn_subr\\lz4\\lz4internal.h' mode='r' encoding='
cp1252'>
Exception ignored in: <_io.FileIO
name='subversion\\libsvn_subr\\utf8proc\\utf8proc_internal.h'
mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='subversion\\libsvn_subr\\utf8proc\\utf8proc_internal.h' mode='r'
 encoding='cp1252'>
Exception ignored in: <_io.FileIO
name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\include\\ap_release.h'
mode='rb' clo
sefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\include\\ap_release.
h' mode='r' encoding='cp1252'>
Exception ignored in: <_io.FileIO
name='C:\\research\\svn\\dev\\deps\\openssl-1.0.2n\\inc32/openssl/opensslv.h'
mode='rb
' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='C:\\research\\svn\\dev\\deps\\openssl-1.0.2n\\inc32/openssl/open
sslv.h' mode='r' encoding='cp1252'>
Exception ignored in: <_io.FileIO
name='C:\\research\\svn\\dev\\deps\\serf-1.3.9\\serf.h' mode='rb'
closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='C:\\research\\svn\\dev\\deps\\serf-1.3.9\\serf.h' mode='r' encod
ing='cp1252'>
Exception ignored in: <function Popen.__del__ at 0x01043420>
Traceback (most recent call last):
  File "C:\Python37\lib\subprocess.py", line 858, in __del__
    ResourceWarning, source=self)
ResourceWarning: subprocess 3716 is still running
Exception ignored in: <function Popen.__del__ at 0x01043420>
Traceback (most recent call last):
  File "C:\Python37\lib\subprocess.py", line 858, in __del__
    ResourceWarning, source=self)
ResourceWarning: subprocess 1356 is still running
'ruby' is not recognized as an internal or external command,
operable program or batch file.
Testing Release configuration on local repository.
-- Running Swig Python tests --
..........................................................Exception
ignored in: <function Popen.__del__ at 0x02D4B030>
Traceback (most recent call last):
  File "C:\Python37\lib\subprocess.py", line 858, in __del__
    ResourceWarning, source=self)
ResourceWarning: subprocess 2972 is still running
...............................................................................................
----------------------------------------------------------------------
Ran 153 tests in 93.884s

OK
]]]

-- 
Johan
Received on 2019-09-29 16:37:11 CEST

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.