[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Problem with control characters in filenames

From: Laurent Saboret <Laurent.Saboret_at_sophia.inria.fr>
Date: 2006-06-26 10:20:15 CEST

Hi Stephan,

Thanks for your reply. See my answers belows.


Stefan Küng a écrit :
> Laurent Saboret wrote:
>> I may have discovered a (minor) bug in TortoiseSVN related to control
>> characters in filenames.
>> The problem is:
>> * Create with Windows Explorer a file named "faa?1" where ? is the
>> ASCII character 1 (type Alt+1). Explorer displays this character as a
>> square.
>> * TortoiseSVN is fine with this filename: add, commit, update, rename
>> and log work fine.
>> * In the other hand, "svn update" breaks svn client on Linux with a
>> message "svn: Can't convert string from 'UTF-8' to native encoding:
>> svn: faa?\226?\152?\1861".
>> Note that the error message is the same when $LANG/$LC_CTYPE are set
>> to fr_FR.iso88591 or en_US.UTF-8.
>> The same problem occur with all control characters from 1 to 31 and 127.
> So this seems to be not a TSVN problem (hey, all TSVN commands work
> fine with that filename) but a Linux/setup problem (or a Subversion
> problem on Linux).
Used alone, svn CL client also works correctly (see my test below). The
problem occurs when both applications are used together.
>> My configuration:
>> The svn server runs svn 1.1.4. I access the repository via svn+ssh.
>> I use as client TortoiseSVN 1.3.4 on Windows NT SP2 and svn 1.3.2 on
>> Linux Fedora Core 3.
>> My guess:
>> According to
>> http://subversion.tigris.org/issues/show_bug.cgi?id=1954, control
>> characters 0x0 to 0x1f and 0x7f are not supported in filenames by SVN
>> (because they cannot be represented in XML) and are rejected by svn
>> client and server >= 1.2.
> AFAIK they're not rejected but 'converted' on the wire to something
> that can be represented in XML.
>> My guess is that TortoiseSVN doesn't check that. As the old svn 1.1.4
> Yes, but so does the Subversion CL client. Ok, you can't really add
> such a file with the CL client, but that's not because of the client
> but because of the console.
I tested svn 1.3.2 CL client on Linux:
./create-test-filenames.pl # Script (attached) that creates filenames
with all ASCII characters from 1 to 255 (except '/')
for f in test* ; do ; svn add $f ; done
=> svn rejects filenames with control characters with an error message
"Invalid control character"
> Unfortunately, Subversion doesn't provide a function to validate a
> path/url before passing one to their API (I've asked for such a
> function, but seems they don't want to implement one).
I had a look to svn 1.3.2 source code. The function may be
svn_path_check_valid() in svn_path.c/.h.
>> server doesn't check it either, a filename with ASCII character 1 can
>> be committed.
>> I assume that the filename gets corrupted in SVN repository, which
>> makes svn Linux client complain about an invalid UTF-8 character.
>> I believe that the problem is minor, thus my SVN server administrator
>> will not agree to upgrade svn.
>> I tried to install a pre-commit hook to reject control characters in
>> filenames. It doesn't work as the filename is already corrupted to
>> "faa??1" where ?? is the wide character 9786.
> So just check for that wide character string - or do you actually use
> that one in a real filename?
Unicode character 9786 (hex 263A) is a smiley, thus is quite unlikely in
a filename ;-) I can certainly test for that character. Thanks.
> Stefan

#!/usr/bin/perl -w

# Create a set of files to test svn capabilities:
# all files of the form "test%ii" for $i in 0..255

use strict;

for my $i (0..255)
    my $filename = "test" . chr($i) . "$i";

    next if (chr($i) eq "/");

    # Create file (with stupid content)
    open(FILE, "> $filename")
        or die "Can\'t open $filename: $!";
    (print FILE "foo\n")
        or die "Can\'t write to $filename: $!";
        or die "Can\'t close $filename: $!";

exit 0;

To unsubscribe, e-mail: users-unsubscribe@tortoisesvn.tigris.org
For additional commands, e-mail: users-help@tortoisesvn.tigris.org
Received on Mon Jun 26 10:25:34 2006

This is an archived mail posted to the TortoiseSVN Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.