On 2020/06/19 22:17, Yasuhito FUTATSUKI wrote:
> On 2020/06/16 2:09, Yasuhito FUTATSUKI wrote:
>> On 2020/06/16 0:18, C. Michael Pilato wrote:
>>> On Mon, Jun 15, 2020 at 10:32 AM Yasuhito FUTATSUKI <futatuki_at_yf.bsdclub.org>
>>> wrote:
>
>>> As for your question: if I force "svnlook" to create errors (by setting
>>> the svnlook variable to "/usr/bin/svn"), today I see an error message with
>>> the b'...' formatting. Adding .decode() as you suggested makes the b'...'
>>> go away and I see what I'd expect to see. As far as I can tell, I'm using
>>> the "en-US.UTF-8" locale, though -- not the "C" one. But maybe I'm just
>>> getting lucky because the locale encoding is UTF-8 and not, say, Shift-JIS
>>> or something? I dunno.
>>
>> I confirmed that the code with .decode() work well in ja_JP.UTF-8 locale
>> on Python 3.6 and Python 3.7 (but I got an error like "'ascii' codec can't
>> decode byte 0xe3 in position 18: ordinal not in range(128)").
> ^ on Python 2.7
>
>> With non UTF-8, non ascii locale, .decode() without specifying encoding
>> causes UnicodeDecodeError on 'utf-8' codecs on Python 3.6 and 3.7.
>> So if we want to allow this script run on non UTF-8, non ascii locale
>> with Python 3, it needs additional code to set encoding.
>
> I wrote "additional code".
>
> [[[
> Index: tools/backup/hot-backup.py.in
> ===================================================================
> --- tools/backup/hot-backup.py.in (revision 1878988)
> +++ tools/backup/hot-backup.py.in (working copy)
> @@ -36,6 +36,7 @@
> ######################################################################
>
> import sys, os, getopt, stat, re, time, shutil, subprocess
> +import locale
> import functools
>
> ######################################################################
> @@ -204,10 +205,18 @@
> """Examine the repository REPO_DIR using the svnlook binary
> specified by SVNLOOK, and return the youngest revision."""
>
> - p = subprocess.Popen([svnlook, 'youngest', '--', repo_dir],
> - stdin=subprocess.PIPE,
> - stdout=subprocess.PIPE,
> - stderr=subprocess.PIPE)
> + if b'' == '':
> + p = subprocess.Popen([svnlook, 'youngest', '--', repo_dir],
> + stdin=subprocess.PIPE,
> + stdout=subprocess.PIPE,
> + stderr=subprocess.PIPE)
> + else:
> + p = subprocess.Popen([svnlook, 'youngest', '--', repo_dir],
> + stdin=subprocess.PIPE,
> + stdout=subprocess.PIPE,
> + stderr=subprocess.PIPE,
> + errors='backslashreplace', # foolproof
> + encoding=locale.getpreferredencoding())
> infile, outfile, errfile = p.stdin, p.stdout, p.stderr
>
> stdout_lines = outfile.readlines()
> @@ -220,7 +229,7 @@
> raise Exception("Unable to find the youngest revision for repository '%s'"
> ": %s" % (repo_dir, stderr_lines[0].rstrip()))
>
> - return stdout_lines[0].strip().decode()
> + return stdout_lines[0].strip()
>
> ######################################################################
> # Main
> ]]]
>
> I confirmed it works on Python 2.7, Python 3.7 on FreeBSD with
> ja_JP.eucJP, ja_JP.UTF-8 locale. However I'm not sure if it works or not
> on Windows, especially non English environment. It depends on that
> the encoding of the error message of SVNLOOK equals to the result of
> locale.getreferredencoding() here.
Although I couldn't confirm it on Windows (because I couldn't set up
gettext/libintl for x64), I commited it in r1882780.
Cheers,
--
Yasuhito FUTATSUKI <futatuki_at_yf.bsclub.org>
Received on 2020-10-23 11:42:08 CEST