Re: Merge policies

On Thu, Apr 19, 2012 at 12:48 PM, Stefan Fuhrmann
<stefanfuhrmann_at_alice-dsl.de> wrote:
> Hi all,
>
> After having a closer look at merge and discussing it
> with Julian on IRC, there seems to be no silver bullet.
> However, we identified a few things that could be changed
> and set of constellations that make merge harder than
> it needs to be.
>
> For the first, there will be another post soon. The second
> boils down to policy. Luckily, SVN has a mechanism to
> enforce policies: server-side hook scripts. My proposal
> is to develop a small set of scripts that a user can
> combine to prevent situations that her life harder than
> necessary. This should give us enough time to improve
> the merge logic inside the SVN libs.
>
> The following pre-commit scripts / policies would be useful.
>
> * Common parts [not a policy]
>  We first check whether the commit contains a changed
>  svn:merge-info property. This limits the performance
>  impact on non-merge commits and we need to identify
>  all changed svn:merge-info anyway.
>
>  Also, the merges that happened on the source branch
>  from a different location than the target branch are
>  of no interest for the policy checkers. E.g.:
>
>  r20: merge r19 from ^/sub-branch to ^/branch
>  txn: merge r10-20 from ^/branch to ^/trunk
>  Both merges will show up in the merge-info delta but
>  we only need to evaluate the second one.

I haven't read your mail in detail yet, but just a note in passing: in
my experience (with a 1.5 server with FSFS-on-NFS backend) propgetting
('svnlook propget') every changed file during pre-commit can be very
expensive. I ran into this with a script for enforcing our autoprops
(like svn:eol-style=native on some files), with a combination of
'svnlook changed' and 'svnlook propget'. On our (admittedly not very
fast) server, users started running into client-side timeouts because
of this, when they committed 300 or so changed files (which happens
easily when syncing branches). The default client-side timeout for
neon is 30 seconds (which can be increased in the 'servers' file), and
if the pre-commit hook takes longer than that, you're screwed.

So I've optimized the pre-commit validation script to only 'propget'
files from 'svnlook changed' which were either Added ('A'), or which
had property modifications ('*U'). So modified files and deleted files
don't have to be propgotten. I suppose this set of pre-commit scripts
can use the same optimization, so it only scales with the amount of
added and prop-changed files/dirs in the transaction.

But even then, someday a user comes along that wants to add 60.000
files (really). He's willing to split it up into commits of say 5000
files each. The pre-commit hook takes approx 2 hours to validate this,
so he has to set his client-side timeout very high while doing this
(after first losing a couple of days before he figures out it's time
to contact an admin).

Maybe some of this is a moot point, because:

- svn+FSFS+NFS might have gotten much faster at getting props??

- maybe propgetting through one of the bindings is much faster than
through svnlook, and you intend for admins to set this up with some
language bindings? In that case, keep in mind that bindings are not
always installed. Svnlook usually is always there.

- maybe you just expect the infrastructure to be much faster?

The problem is mainly that 'svnlook propget' doesn't support recursive
propgetting, nor getting all props from the entire transation, or ...
only the props for a single path. Which means you have to invoke it
separately for every item that needs to be inspected. Maybe that can
be improved somehow?

Anyway, these are just some slightly OT random thoughts.

-- 
Johan