[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Unicode UTF-16 files detected as binary

From: Peter N. Lundblad <peter_at_famlundblad.se>
Date: 2005-01-07 08:17:52 CET

On Fri, 7 Jan 2005, Joel wrote:

> > [...]
> > > All of the systems I work with on a regular basis (Mac, Linux, fBSD,
> > > MSWxxx) handle Unicode Japanese, and include a default GUI text editor
> > > that is usable for many of the large character set languages. Many of
> > > the terminal programs can handle Unicode Japanese if you set them up to
> > > do so. (Install the fonts and export LANG=ja_JP.UTF-8 or something like
> > > that.) You can get VIM and emacs set up to handle Unicode, as well, if
> > > you look/ask around. I've used Netbeans with Japanese, so I know that
> > > works, and I understand that Eclipse does well, also.
> > >
> > Please don't confuse Unicode and UTF16. Unicode is a character repertoire.
>
> I was trying to avoid a pedantic tone. (Bad hair day yesterday, and I
> didn't want to take it out on a mailing list. :-/) But, yes, the
> distinction should be made clear.
>
Many people still don't know that Unicode and UTF16 isn't equivalent and
then start claiming that someone doesn't support Unicode because he
doesn't support UTF16.

> > UTF16 is a way to serialize a stream of characters from that Repertoire.
> > Subversion supports Unicode, but it doesn't handle UTF16 very well yet.
> > So, there is no problem using Unicode with subversion, just use UTF8.
>
> From someone's comments, it sounded as if multibyte UTF-8 characters
> would be handled as binary. That would indicate that the support is
> not really complete for UTF-8.
>
Maybe I confused things by saying something like if you say UTF16 is
binary, then you could say that about UTF8 as well if your tool doesn't
have the adequate support. That's not the case for svn, though.

> > That being said, I agree that we should support the other transformation
> > forms as well.
>
> I'd be happy to offer free advice to those who would be working on it.
> (Of course, I can't guarantee my advice would be worth what it costs,
> but even having someone to bounce ideas off can help.) I suppose I
> should make this offer on the dev list ...
>
Input is always welcome.

Thanks,
//Peter

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Fri Jan 7 08:32:00 2005

This is an archived mail posted to the Subversion Users mailing list.