[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

RFC: building eol and encoding support into svn_stream_t

From: Peter N. Lundblad <peter_at_famlundblad.se>
Date: 2004-10-17 23:39:12 CEST


While svn has been localized, several encoding bugs have been found, we
have some open that are hard to solve, I'm sure we will introduce new ones
and there are probably more that we haven't found yet.

I think this is partly because we have different ways of recoding and
end-of-line translating text in different places:
- we have svn_cmdline_{printf,puts,...} that take arguments in UTF-8 and
output in console/locale encoding and native eol-style.
- we have some (no many) direct C stdio calls left in the code for various
reasons. They do eol-style translation, but expect their arguments to be
in the correct encoding.
- When we write to svn_stream_t* objects or apr_file_t*, we have to do
recoding and eol-translation ourselves, which is messy and error-prone. It
is also problematic since we have different encodings for stdout (the
console) and other files on some platforms, i.e. Windows. This is a
problem since a function that produces output needs to know if the output
is for a console or a file. This is the case for svn_client_diff. Also,
some functions need to turn off recoding for parts of their output.
Examples are proplist -v and diff where we might not know the encoding of
the propval or the file contents.

I think this is a real mess that we have to fix in some way.

My proposal is to add support for recoding and eol translation to our
svn_stream_t interface. Then we should use streams instread of apr_file_t
whereever possible, such as in svn_client_diff. It should be possible to
set the encoding for a stream and turn it into text/binary mode.

Why not just stack a recoding stream onto an existing stream like we do
for compression? That's because the creator of the stream knows the
encoding it should be outputing, but the user of the stream may need to
turn it into binary mode.

Since the stream objects are opaque to the outside world, I think we can
add encoding information to it without breaking any compability.

I'd like to get this done for 1.2, so we can get correct and consistent
output from our commands on all platforms.



To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sun Oct 17 23:24:32 2004

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.