Re: Python3 support for hot-backup.py

On 2020/06/16 2:09, Yasuhito FUTATSUKI wrote:
> On 2020/06/16 0:18, C. Michael Pilato wrote:
>> On Mon, Jun 15, 2020 at 10:32 AM Yasuhito FUTATSUKI <futatuki_at_yf.bsdclub.org>
>> wrote:

>> As for your question: if I force "svnlook" to create errors (by setting
>> the svnlook variable to "/usr/bin/svn"), today I see an error message with
>> the b'...' formatting. Adding .decode() as you suggested makes the b'...'
>> go away and I see what I'd expect to see. As far as I can tell, I'm using
>> the "en-US.UTF-8" locale, though -- not the "C" one. But maybe I'm just
>> getting lucky because the locale encoding is UTF-8 and not, say, Shift-JIS
>> or something? I dunno.
>
> I confirmed that the code with .decode() work well in ja_JP.UTF-8 locale
> on Python 3.6 and Python 3.7 (but I got an error like "'ascii' codec can't
> decode byte 0xe3 in position 18: ordinal not in range(128)"). ^ on Python 2.7
 
> With non UTF-8, non ascii locale, .decode() without specifying encoding
> causes UnicodeDecodeError on 'utf-8' codecs on Python 3.6 and 3.7.
> So if we want to allow this script run on non UTF-8, non ascii locale
> with Python 3, it needs additional code to set encoding.

I wrote "additional code".

[[[
Index: tools/backup/hot-backup.py.in
===================================================================
--- tools/backup/hot-backup.py.in (revision 1878988)
+++ tools/backup/hot-backup.py.in (working copy)
@@ -36,6 +36,7 @@
 ######################################################################
 
 import sys, os, getopt, stat, re, time, shutil, subprocess
+import locale
 import functools
 
 ######################################################################
@@ -204,10 +205,18 @@
   """Examine the repository REPO_DIR using the svnlook binary
   specified by SVNLOOK, and return the youngest revision."""
 
- p = subprocess.Popen([svnlook, 'youngest', '--', repo_dir],
- stdin=subprocess.PIPE,
- stdout=subprocess.PIPE,
- stderr=subprocess.PIPE)
+ if b'' == '':
+ p = subprocess.Popen([svnlook, 'youngest', '--', repo_dir],
+ stdin=subprocess.PIPE,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE)
+ else:
+ p = subprocess.Popen([svnlook, 'youngest', '--', repo_dir],
+ stdin=subprocess.PIPE,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ errors='backslashreplace', # foolproof
+ encoding=locale.getpreferredencoding())
   infile, outfile, errfile = p.stdin, p.stdout, p.stderr
 
   stdout_lines = outfile.readlines()
@@ -220,7 +229,7 @@
     raise Exception("Unable to find the youngest revision for repository '%s'"
                     ": %s" % (repo_dir, stderr_lines[0].rstrip()))
 
- return stdout_lines[0].strip().decode()
+ return stdout_lines[0].strip()
 
 ######################################################################
 # Main
]]]

I confirmed it works on Python 2.7, Python 3.7 on FreeBSD with
ja_JP.eucJP, ja_JP.UTF-8 locale. However I'm not sure if it works or not
on Windows, especially non English environment. It depends on that
the encoding of the error message of SVNLOOK equals to the result of
locale.getreferredencoding() here.

Cheers,

-- 
Yasuhito FUTATSUKI <futatuki_at_yf.bsclub.org>