Re: Huge diff for one-line change in ASF repository

On Fri, Oct 14, 2011 at 4:56 PM, Daniel Shahaf <d.s_at_daniel.shahaf.name> wrote:
> Johan Corveleyn wrote on Fri, Oct 14, 2011 at 16:25:46 +0200:
>> These "modified" files can also cause tree conflicts (which can
>> generate quite some WTF's).
>>
>
> Tree conflicts?  How?  I'd have expected (full-file) text conflicts only.

Ok, maybe tree conflicts are more rare, but they can happen, of the
kind "locally modified, incoming delete". We experienced one of those,
because one of the corrupt ("modified") files was inside a directory
that was deleted upon update (or merge, I don't remember). I can
assure you that this raises quite some eyebrows :-) (what local
modification? I haven't touched that file!)

>> You can identify the corrupt files easily by taking a look at the
>> corresponding pristine files (or .svn-base files in .svn/text-base).
>> They will have CRLF, while "correct" files with eol-style native
>> should only have LF in the pristine file.
>>
>
> Nice.  I did wonder about that, since 'svn cat' silently transform the EOLs.
>
>> This makes me wonder:
>> - Why can't the server verify that files with native eol-style are
>> committed to the repository with LF line-termination? That would seem
>> like a very sane validation.
>
> It's possible.  If we do this we'll (a) cause files in existing
> repositories to be uncommittable by tools that don't convert to LF
> before committing (this does _not_ include the 'svn' client, and
> probably doesn't include anything that goes through libsvn_client),
> (b) probably get some flames along the lines of "The svn devs broke
> git-svn on purpose". :P

Ah, but git-svn doesn't seem to honor the invariant that files with
native eol-style *must* be converted to LF by the client. By not doing
this, they break things for every other svn client using the same
repository. So this is clearly a bug in git-svn. Of course, to avoid
flames, we should probably first escalate the problem to the git-svn
people, to get this bug fixed (or maybe it is already fixed?).

I think it would be a good thing if the server protected against bugs
in clients by checking this invariant (in the spirit of "don't trust
anything that comes from the client").

>> - Currently the client is the sole responsible to do this, but a
>> broken client can break a lot of things for all other users.
>
> Yep.  And along the same lines, the client may want to force EOL
> canonicalization in code paths that currently just assume that the
> server will provide files with the correct EOL styles.

Maybe that's also a possible solution. Though I'm wondering if this
would be sufficient: you can also see the problem just by diffing
between repository URL's / revisions (if you "svnlook diff -c REV" one
of those commits, you'll get a full-file diff, although the eol-style
property wasn't changed; that's not nice). Of course, one could make
"svnlook diff" etc. compensate for this too, but that really feels as
some sort of repository corruption which we're papering over.

>> - Maybe as a workaround: can someone come up with a hook script for
>> checking this situation in a pre-commit hook?
>>
>
> for f in $CHANGED_FILES; do
>  if svnlook pl -t $txn $repos $f | grep -w svn:eol-style >/dev/null && [ "`svnlook cat -t $txn $repos $f | xxd -ps -c1 | fgrep 0d | head -n1`" ]; then
>    exit 1
>  fi
> done

Thanks. Seems I can experiment a bit with that. Though I think the
problem is specific to eol-style=native (not any eol-style). I thought
that the "conversion-to-LF" only happens for native, and not for
svn:eol-style=LF or svn:eol-style=CRLF (I thought in that case the LF,
or CRLF was saved "as is" in the repository). But I haven't checked
that (we only use native).

-- 
Johan