[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Unicode characters in filenames on windows

From: ВарфоломеевИгорь <i3v_at_mail.ru>
Date: Wed, 12 Jun 2013 23:18:36 +0000 (UTC)

Branko Čibej <brane <at> wandisco.com> writes:

>
> On 12.06.2013 02:58, Варфоломеев Игорь wrote:
> > Hi all,
> > I'm still not sure if it's a bug, or if I'm doing something wrong. But
I'm unable to get TortoiseSVN
> > command-line tool to work with files with UTF-8 characters in their name.
> > (1.7.10 r1485443, part of TortoiseSVN 1.7.13 <at> Win7 x64)
> >
> >
> > I've posted the following message here
> > (
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=3057388
)
> > but was suggested
> > (
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=3057494
)
> > to re-address it to users <at> subversion.apache.org :
> >
> >
> >
---------------------------------------------------------------------------------
> > *** THE ISSUE ***
> >
> >
> > Workflow:
> >
> > 1. Create "c:\temp\UNCtest\R_UNCtest\" folder
> > 2. Create a repository with default file structure in it
> > 3. Checkout "trunk" dir to "c:\temp\WC\trunk"
> > 4. Create file "c:\temp\WC\trunk\1‐2.txt" ,
> > note, that filename consists of 3 symbols, and the one in the middle is
"HYPHEN" or ‐, (see
> http://www.fileformat.info/info/unicode/char/2010/index.htm )
> >
> > 5. Add and commit this file with Tortoise GUI.
> > (this works OK)
> > 6. start windows cmd
> > 7. make sure your cmd is set to use UTF-8 compatible font, for example,
"Consolas" (see
>
http://stackoverflow.com/questions/10764920/utf-16-on-cmd-exe/10765469#10765469
).
> > 8. navigate to "c:\temp\WC\trunk"
> > 9. type "dir" - you should see the listing correctly, including
"1‐2.txt" file
> > 10. Type "mkdir 1‐2" - this should correctly create a directory.
> > 11. Type "svn info 1‐2.txt"
> > Result:
> > --------------------------------------------------
> > svn: warning: W155010: The node 'C:\TEMP\UNCtest\WC\trunk\1?2.txt' was
not found
> > .
> >
> > svn: E200009: Could not display info for all targets because some
targets don't
> > exist
>
> I believe this happens because the "chcp" command changes the OEM
> (console) code-page, but Subversion uses the ANSI (Windows) code page
> for input and output conversion. In other words, the "chcp" does not
> affect the command-line client in any way.
>
> > * Am I doing something wrong?
>
> Not as such. :)
>
> > * Or could this situation be treated as a bug?
> > * Or, maybe a “feature request”?
>
> I think the only way to actually get this right is to change the way we
> read from and write to the console on Windows. Instead of converting
> strings to some native encoding and using the ordinary output functions,
> we should convert to UTF-16 and use wide-char output functions instead.
>

I'm still not sure what should I do next...
Should I re-address this to dev_at_subversion.apache.org ?

Anyway, here's a batch file, reproducing the issue:
https://skydrive.live.com/redir?resid=8FD4EC3161ABA67A!916

==========================================================================
@echo off

rem * This batch file aims to reproduce an issue with Unicode support in svn
CLI.
rem * This file is in UTF-8 65001 encoding, and it should not be converted
to ANSI or etc.
rem * I highly recommend to change default cmd font to "Consolas", for
Unicode support,
rem see
http://stackoverflow.com/questions/10764920/utf-16-on-cmd-exe/10765469#10765469
rem * This file is based on template from
rem
http://subversion.apache.org/docs/community-guide/issues.html#reporting-bugs
rem
rem *** TESTED ON ***
rem Win7 x64 - working
rem WinXP x64 - not working (unable to create a file with valid name)
rem 1.7.10 r1485443, part of TortoiseSVN 1.7.13 x64
rem

:defineCommands
rem You might need to adjust these lines to point to your
rem compiled-from-source Subversion binaries, if using those:
for %%i in (svn.exe) do set SVN="%%~$PATH:i"
for %%i in (svnadmin.exe) do set SVNADMIN="%%~$PATH:i"

:defineUrls
rem Only supported access method: file://. If http:// or svn://, then
rem you'll have to configure it yourself first.
set URL=file:///%CD%/repos
set URL=%URL:\=/%
echo Base url for repo: %URL%

:cleanAllDirsAndCreateRepo
if exist repos rmdir /s /q repos
if exist import-me rmdir /s /q import-me
if exist wc rmdir /s /q wc
%SVNADMIN% create repos

:prepareGreekTree
echo Making a Greek Tree for import...
mkdir import-me
mkdir import-me\trunk
mkdir import-me\tags
mkdir import-me\branches

rem -------------------------------------------------------------
rem HERE WE GO:

chcp 65001

rem Note: Some people say, that "chcp 65001" on a line by itself aborts the
batch file.
rem And this seem to be true for Windows XP x64. Though, it works OK on Win7
x64.
rem see
http://stackoverflow.com/questions/2182568/batch-script-is-not-executed-if-chcp-was-called/2462138#2462138
rem If you're experiencing this issue, you may simply type "chcp 65001" before
rem launching this script.

echo This is the file '1‐2.txt'. > import-me\trunk\1‐2.txt

rem Note1: The symbol between "1" and "2" is "HYPHEN" or ‐,
rem see http://www.fileformat.info/info/unicode/char/2010/index.htm
rem If you're getting incorrect filename, like "1тАР2.txt" (or something)
rem instead of "1‐2.txt", make sure you've typed "chcp 65001".
rem
rem Note2: "import-me\trunk\1‐2.txt" itself would have 65001 encoding too.

echo Importing it...
cd import-me
%SVN% import -q -m "Initial import." %URL%
cd ..

:checkoutTrunk
echo Checking out working copy..
%SVN% co -q %URL%/trunk wc

echo Here's the Error1:
%SVN% info wc/1‐2.txt

rem | svn: warning: W155010: The node 'F:\temp\wc\1?2.txt' was not found.
rem |
rem | svn: E200009: Could not display info for all targets because some
targets don't
rem | exist

echo This is a modification. >> wc\1‐2.txt

echo Here's the Error2:
%SVN% commit wc\1‐2.txt
rem |svn: E200009: Commit failed (details follow):
rem |svn: E200009: 'F:\temp\wc\1?2.txt' is not under version control

rem I've not tested all svn commands.. there may be others...

goto :eof

==========================================================================

Igor
Received on 2013-06-13 01:20:36 CEST

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.