Re: converting unconvertible UTF-8 data

On Sun, 2002-07-21 at 20:21, Karl Fogel wrote:

> That's not the only problem -- the portability issue in your previous
> paragraph is the real showstopper.

I wouldn't say so. At compile time you can have found out whether the
implementation supports //TRANSLIT or not. Just don't enable it on
platforms where it's not available.

And no, //TRANSLIT is not ignored where it is not recognized. The use
of slashes to separate fields in the name is something which I've
introduced and only the GNU libiconv uses it as well (at least to the
best of my knowledge).

> Either way, we may still eventually want our own fuzzy function to
> supply whatever cannot be depended on from iconv. It's good if
> Subversion behaves as close to the same everywhere as possible.

When it comes to these things it really a quality of implementation
thing. You don't want to drag down to quality on one system just
because there are some others which don't have implementations of the
needed functionality.

Also, consistency in the output of information which is meant for the
user (e.g., log files, changelogs etc) is not that important. A human
reader is able to figure out a lot despite noise on the channel.

> And we can eventually give our "fuzzy" function the option of doing
> transliteration. But I think the initial implementation would better
> output ?\XXX for each unconverted byte, since that's simple to get
> right initially.

If you do this (and it's not easy if you use iconv for the conversion),
why not enable the transliteration for the platforms which support it?
All you need is a configure test and a bit of #ifdef'ed code to append
"//TRANSLIT" to the codeset name.

-- 
---------------.                          ,-.   1325 Chesapeake Terrace
Ulrich Drepper  \    ,-------------------'   \  Sunnyvale, CA 94089 USA
Red Hat          `--' drepper at redhat.com   `------------------------