On 06.01.2019 19:54, Daniel Shahaf wrote:
> Branko Čibej wrote on Sun, 06 Jan 2019 19:37 +0100:
>> A simple check would be:
>>
>> * if 0x0a is on an odd offset, and the next byte is 0x00, then it's a
>> UTF-16-LE linefeed;
>> * else if 0x0a is on an even offset, and the _previous_ byte is 0x00,
>> then it's a UTF-16-BE linefeed;
> Would would happen if it were an ASCII/UTF-8 file that happened to
> have a literal NUL byte next to an LF byte? I have seen/used
> some of those.
Yes, well, in that case the NUL just might randomly "move" from one line
to the next, depending on changes in the file. Nothing we can do
short-term will fix that ... and as far as I'm concerned that's not a
valid text file, so it won't disturb me too much if blame results are a
bit fuzzy in such cases.
>> * otherwise just hope it's a linefeed and move on.
> The encoding may also be set explicitly via a svn:mime-type="text/foo;
> charset=utf-16-le" property.
Yes, but supporting _that_ in our code is a much, much bigger task than
what we're talking about in this thread. What I proposed here would be a
simple, localised hack to avoid splitting lines in the middle of a code
sequence. If we do decide to implement something like that for line
breaks, we should extend it to both variants of UTF-32 as well.
(Also, note the abuse of the 'charset' attribute in Content-Type ... the
"character set" is Unicode, it should be Content-Encoding: UTFsomething,
but that's used for something different again :\ )
> (We even parse that in mod_dav_svn, I think?)
Even if we do, it's irrelevant to this discussion. We set the value of
the Content-Type header based on svn:mime-type for a simple GET request,
but don't interpret it otherwise on the server side.
-- Brane
Received on 2019-01-07 05:56:06 CET