Re: [l10n] Problem: RA via svnserve tunnel mode always sends back localized error messages?

Hi,

From: "Peter N. Lundblad" <peter@famlundblad.se>
Subject: Re: [l10n] Problem: RA via svnserve tunnel mode always sends back localized error messages?
Date: Tue, 29 Mar 2005 14:12:09 +0200 (CEST)

> > I get funny error messages when using svnserve tunnel mode with LANG=C
> > (or LC_ALL=C).
> > With LANG=C, error message characters should be ASCII.
> > In fact, however, they seem to be always localized UTF-8 characters,
> > and when displayed under LANG=C they turn to unreadable '?\nnn'-s, that
> > shows they are recoded twice from UTF-8 to local encoding.
>
> When the ctype locale category is set to "C", libc will think your
> environment only supports pure ASCII. So, the above is not caused by
> double decoding, but iconv is trying to recode your (Japanese?) characters
> from UTF8 to (7-bit) ASCII.

I see... Thanks for explanation.

> > This strange behaviour occurs in both /branches/1.1.x and current /trunk.
> >
> > A test shell script and its outputs are in the lower part of this mail.
> > In that output, the error message "File not found: revision %ld, path '%s'"
> > (in subversion/libsvn_fs_*/tree.c) is:
> >
> > * properly localized with LANG=ja_JP.eucJP + ra_local.
> > * output in ASCII with LANG=C + ra_local.
> > * properly localized with LANG=ja_JP.eucJP + ra_svn (tunnel mode).
> > * output in unreadable chars ('?\nnn'-s) with LANG=C + ra_svn (tunnel mode).
> The problem here is that svnserve sets the locale according to the
> environment on the server (it has to, to handle characters correctly).
> Does your login script on the server set the locale? I'm not sure how SSH
> transfers locale envvars over the wire.

I'm using Zsh and I did my previous test with $LANG set as 'ja_JP.eucJP'
at .zshrc.
So, this time I made another test:

* Remove $LANG settings from .zshrc.
* Print locale settings at not_found in subversion/libsvn_fs_fs/tree.c
  and main at subversion/svnserve/main.c.
  Code are such like this:
    fputs("In svnserve/main.c/main, ", stderr);
    getenv("LC_ALL")
      ? fprintf(stderr, "$LC_ALL=%s\n", getenv("LC_ALL"))
      : getenv("LANG")
          ? fprintf(stderr, "$LANG=%s\n", getenv("LANG"))
          : fprintf(stderr, "$LANG=%s\n", "locale?");
* (Refactor my test script.)

The result shows a curious fact (details are in the lower part of this mail):

Via svn+ssh, $LANG is always set as 'ja_JP.EUC-JP',
whether svn is executed with $LANG set as 'C' or as 'ja_JP.eucJP'.
(Note: 'ja_JP.EUC-JP' and 'ja_JP.eucJP' are the same locale.
 However, there is no 'ja_JP.EUC-JP' within my homedir condig files!)

What is handling locales with my repository access via svn+ssh?

For the record, my environments are:

* OS: Debian GNU/Linux 3.0r4
* SSH: OpenSSH_3.4p1 Debian 1:3.4p1-1.woody.3, SSH protocols 1.5/2.0,
       OpenSSL 0x0090603f

> See also recent commits by eh regarding locale stuff in svnserve.

OK, I'll check notes/l10n-problems.

--
[test script]
#!/bin/sh
repos_path=$HOME/tests/svnssh/repos
repos_uri_file=file://$repos_path
repos_uri_svnssh=svn+ssh://127.0.0.1$repos_path
wc_path_file=$HOME/tests/svnssh/wc_file
wc_path_svnssh=$HOME/tests/svnssh/wc_svnssh
obj1=$wc_path_file/obj1
obj2=$wc_path_file/obj2
obj2_svnssh=$wc_path_svnssh/obj2
svnadmin=./subversion/svnadmin/svnadmin
svn=./subversion/clients/cmdline/svn
$svnadmin create $repos_path
$svn co -q $repos_uri_file $wc_path_file
touch $obj1
$svn add -q $obj1
$svn ci -q -m "Add obj1." $obj1 # r1
$svn move -q $obj1 $obj2
$svn ci -q -m "Rename obj1 to obj2." $obj2 # r2
$svn co -q $repos_uri_svnssh $wc_path_svnssh
echo @ LANG=ja_JP.eucJP, via file
LANG=ja_JP.eucJP $svn log -q -r1 $obj2 # OK.
echo @ LANG=C, via file
LANG=C $svn log -r1 $obj2 # OK.
echo @ LANG=ja_JP.eucJP, via svn+ssh
LANG=ja_JP.eucJP $svn log -q -r1 $obj2_svnssh # OK.
echo @ LANG=C, via svn+ssh
LANG=C $svn log -r1 $obj2_svnssh # '?\nnn'-s!!
rm -rf $repos_path $wc_path_file $wc_path_svnssh
[result]
nori1[2:15]%  ./svnsshtest.sh                      saba:~/svnwc/subversion-head
In subversion/libsvn_fs_fs/tree.c/not_found, $LANG=ja_JP.eucJP
nori1@127.0.0.1's password:
In svnserve/main.c/main, $LANG=ja_JP.EUC-JP
nori1@127.0.0.1's password:
In svnserve/main.c/main, $LANG=ja_JP.EUC-JP
nori1@127.0.0.1's password:
In svnserve/main.c/main, $LANG=ja_JP.EUC-JP
@ LANG=ja_JP.eucJP, via file
In subversion/libsvn_fs_fs/tree.c/not_found, $LANG=ja_JP.eucJP
In subversion/libsvn_fs_fs/tree.c/not_found, $LANG=ja_JP.eucJP
svn: ファイルが見つかりません: リビジョン 1, パス '/obj2'
@ LANG=C, via file
In subversion/libsvn_fs_fs/tree.c/not_found, $LANG=C
In subversion/libsvn_fs_fs/tree.c/not_found, $LANG=C
svn: File not found: revision 1, path '/obj2'
@ LANG=ja_JP.eucJP, via svn+ssh
nori1@127.0.0.1's password:
In svnserve/main.c/main, $LANG=ja_JP.EUC-JP
In subversion/libsvn_fs_fs/tree.c/not_found, $LANG=ja_JP.EUC-JP
In subversion/libsvn_fs_fs/tree.c/not_found, $LANG=ja_JP.EUC-JP
svn: ファイルが見つかりません: リビジョン 1, パス '/obj2'
@ LANG=C, via svn+ssh
nori1@127.0.0.1's password:
In svnserve/main.c/main, $LANG=ja_JP.EUC-JP
In subversion/libsvn_fs_fs/tree.c/not_found, $LANG=ja_JP.EUC-JP
In subversion/libsvn_fs_fs/tree.c/not_found, $LANG=ja_JP.EUC-JP
svn: ?\227?\131?\149?\227?\130?\161?\227?\130?\164?\227?\131?\171?\227?\129?\140?\232?\166?\139?\227?\129?\164?\227?\129?\139?\227?\130?\138?\227?\129?\190?\227?\129?\155?\227?\130?\147: ?\227?\131?\170?\227?\131?\147?\227?\130?\184?\227?\131?\167?\227?\131?\179 1, ?\227?\131?\145?\227?\130?\185 '/obj2'
Thanks,
-- 
|:  Noritada KOBAYASHI
|:  Dept. of General Systems Studies,
|:  Graduate School of Arts and Sciences, Univ. of Tokyo
|:  E-mail: nori1@dolphin.c.u-tokyo.ac.jp (preferable)
|:          nori@esa.c.u-tokyo.ac.jp
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org