[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Problems running testsuite on Windows with Python 3

From: Yasuhito FUTATSUKI <futatuki_at_poem.co.jp>
Date: Thu, 23 Apr 2020 02:05:16 +0900

On 2020/04/22 23:04, Johan Corveleyn wrote:
> On Wed, Apr 22, 2020 at 3:50 PM Yasuhito FUTATSUKI <futatuki_at_poem.co.jp> wrote:
 
>> ... and what is worse, at least
>>
>>> FAIL: merge_tests.py 34: conflict markers should match the file's eol style
>>
>> this one seems to be broken even with Python 2.7, on Windows.
>> (I'll post about it later).
>
> That's strange. It does succeed on my system when running with Python
> 2.7.17. I had "All successful" test runs for [fsfs] x [ra_local,
> ra_serf, ra_svn].

Yes, I don't doubt this test is passed on Windows with Python 2.7,
but I doubt this test does not check just what we want to check.

As far as I read the description of the test and about KEEP_EOL_STYLE
option introduced in r1743445.

merge_tests.py (merge_conflict_markers_matching_eol):
[[[
# eol-style handling during merge with conflicts, scenario 1:
# when a merge creates a conflict on a file, make sure the file and files
# r<left>, r<right> and .mine are in the eol-style defined for that file.
]]]

So I think comparion of expected and actual should be done without
eol-style translation. However it try to check with translation on
Windows environment.

[[[
  # CRLF is a string that will match a CRLF sequence read from a text file.
  # ### On Windows, we assume CRLF will be read as LF, so it's a poor test.
  if os.name == 'nt':
    crlf = '\n'
  else:
    crlf = '\r\n'

  # Strict EOL style matching breaks Windows tests at least with Python 2
  keep_eol_style = not svntest.main.is_os_windows()

    ...

  # do the test for each eol-style
  for eol, eolchar in zip(['CRLF', 'CR', 'native', 'LF'],
                          [crlf, '\015', '\n', '\012']):

    ...

    svntest.actions.run_and_verify_merge2(wc_backup, cur_rev - 1, cur_rev,
                                          sbox.repo_url, None,
                                          expected_backup_output,
                                          expected_mergeinfo_output,
                                          expected_elision_output,
                                          expected_backup_disk,
                                          expected_backup_status,
                                          expected_backup_skip,
                                          keep_eol_style=keep_eol_style)
]]]

So, svntest.actions.run_and_verify_merge2 is called with
keep_eol_style=False.

The docstring of svntest.actons.run_andverify_merge2 said:
[[[
  """
    ...

  If KEEP_EOL_STYLE is set, don't let Python normalize the EOL when
  reading working copy contents as text files. It has no effect on
  binary files.

    ...
  """
]]]

It seems that this works only for Python 3. Even KEEP_EOL_STYLE=False,
Python 2.7 on Windows still read '\r' as it is, as I didn't expect
when I read it first time.

To make it work as I expected (and just work as Python 3 do), apply
this patch:
[[[
Index: subversion/tests/cmdline/svntest/wc.py
===================================================================
--- subversion/tests/cmdline/svntest/wc.py (revision 1876827)
+++ subversion/tests/cmdline/svntest/wc.py (working copy)
@@ -28,6 +28,7 @@
 import re
 import logging
 import pprint
+import io
 
 if sys.version_info[0] >= 3:
   # Python >=3.0
@@ -686,9 +687,9 @@
         if os.path.isfile(node):
           try:
             if keep_eol_style:
- contents = open(node, 'r', newline='').read()
+ contents = io.open(node, 'r', newline='').read()
             else:
- contents = open(node, 'r').read()
+ contents = io.open(node, 'r').read()
           except:
             contents = open(node, 'rb').read()
         else:
]]]
and this will make merge_tests.merge_conflict_markers_matching_eol()
and perhaps some other tests fail on Windows with Python 2.7.

a fix for merge_tests.merge_conflict_markers_matching_eol and
merge_tests.merge_eolstyle_handling will be:
[[[
Index: subversion/tests/cmdline/merge_tests.py
===================================================================
--- subversion/tests/cmdline/merge_tests.py (revision 1876827)
+++ subversion/tests/cmdline/merge_tests.py (working copy)
@@ -3323,16 +3323,12 @@
 
   mu_path = sbox.ospath('A/mu')
 
- # CRLF is a string that will match a CRLF sequence read from a text file.
- # ### On Windows, we assume CRLF will be read as LF, so it's a poor test.
   if os.name == 'nt':
- crlf = '\n'
+ native_nl = '\r\n'
   else:
- crlf = '\r\n'
+ native_nl = '\n'
+ crlf = '\r\n'
 
- # Strict EOL style matching breaks Windows tests at least with Python 2
- keep_eol_style = not svntest.main.is_os_windows()
-
   # Checkout a second working copy
   wc_backup = sbox.add_wc_path('backup')
   svntest.actions.run_and_verify_svn(None, [], 'checkout',
@@ -3349,8 +3345,8 @@
   path_backup = os.path.join(wc_backup, 'A', 'mu')
 
   # do the test for each eol-style
- for eol, eolchar in zip(['CRLF', 'CR', 'native', 'LF'],
- [crlf, '\015', '\n', '\012']):
+ for eol, eolchar in zip(['CRLF', 'CR','native', 'LF'],
+ [crlf, '\015', native_nl, '\012']):
     # rewrite file mu and set the eol-style property.
     svntest.main.file_write(mu_path, "This is the file 'mu'."+ eolchar, 'wb')
     svntest.main.run_svn(None, 'propset', 'svn:eol-style', eol, mu_path)
@@ -3445,7 +3441,7 @@
                                           expected_backup_disk,
                                           expected_backup_status,
                                           expected_backup_skip,
- keep_eol_style=keep_eol_style)
+ keep_eol_style=True)
 
     # cleanup for next run
     svntest.main.run_svn(None, 'revert', '-R', wc_backup)
@@ -3468,16 +3464,8 @@
 
   mu_path = sbox.ospath('A/mu')
 
- # CRLF is a string that will match a CRLF sequence read from a text file.
- # ### On Windows, we assume CRLF will be read as LF, so it's a poor test.
- if os.name == 'nt':
- crlf = '\n'
- else:
- crlf = '\r\n'
+ crlf = '\r\n'
 
- # Strict EOL style matching breaks Windows tests at least with Python 2
- keep_eol_style = not svntest.main.is_os_windows()
-
   # Checkout a second working copy
   wc_backup = sbox.add_wc_path('backup')
   svntest.actions.run_and_verify_svn(None, [], 'checkout',
@@ -3518,7 +3506,7 @@
                                         expected_backup_disk,
                                         expected_backup_status,
                                         expected_backup_skip,
- keep_eol_style=keep_eol_style)
+ keep_eol_style=True)
 
   # Test 2: now change the eol-style property to another value and commit,
   # merge this revision in the still changed mu in the second working copy;
@@ -3549,7 +3537,7 @@
                                         expected_backup_disk,
                                         expected_backup_status,
                                         expected_backup_skip,
- keep_eol_style=keep_eol_style)
+ keep_eol_style=True)
 
   # Test 3: now delete the eol-style property and commit, merge this revision
   # in the still changed mu in the second working copy; there should be no
@@ -3578,7 +3566,7 @@
                                         expected_backup_disk,
                                         expected_backup_status,
                                         expected_backup_skip,
- keep_eol_style=keep_eol_style)
+ keep_eol_style=True)
 
 #----------------------------------------------------------------------
 def create_deep_trees(wc_dir):
]]]

Or alternatively we can change behavior about KEEP_EOL_STYLE option
on Python 3 to just same as it is on Python 2.7 (I don't want to do so,
though).

Cheers,

-- 
Yasuhito FUTATSUKI <futatuki_at_yf.bsdclub.org> / <futatuki_at_poem.co.jp>
Received on 2020-04-22 19:08:06 CEST

This is an archived mail posted to the Subversion Dev mailing list.