[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Scalability issue with merges in busy projects

From: Branko Čibej <brane_at_wandisco.com>
Date: Wed, 15 Jan 2014 13:27:22 +0100

On 15.01.2014 13:11, Stefan Sperling wrote:
> On Tue, Jan 14, 2014 at 11:00:12PM +0100, Stefan Fuhrmann wrote:
>> Hi all,
>> We do have a "usage race" in SVN that I will commit
>> a fix to in a few minutes. This is mainly to document
>> the problem and maybe get feedback in case I over-
>> looked something.
>> Situation:
>> * We track merges in properties at the top-level dir
>> (usually the branch / trunk root). To commit a merge,
>> the user must update to HEAD and then commit.
>> TXNs based on older revs will be rejected by
>> lib_repos/commit.c: change_dir_prop().
>> * Transmitting the merge result may take minutes in
>> the case of a large merge (many files, large artwork
>> etc.).
>> * The commit will still fail if there has been another
>> commit on the merge target while merge commit
>> was transmitted.
>> Problem:
>> As a result, the average time between commits to the
>> target branch limits the size of data you may merge
>> (and successfully commit).
>> Solution:
>> We can't do much about the first and second points
>> in the list above but wc updates after small commits
>> should be much faster. i.e. much less of a limitation.
>> The conflict handling in our commit code will be
>> extended such that simple bubble-up directory changes
>> merge cleanly with property changes on that directory.
>> We verify that the entries in the target directory are
>> still the same as in the txn base modulo some version
>> bumps. That way, the working copy will look like the
>> directory being up-to-date and some of the sub-dirs
>> being updated to an older revision.
>> -- Stefan^2.
> I'm not sure I fully understand what you are proposing to do.
> But whatever you do, please don't auto-merge deletions of already
> deleted items. Doing so prevents some tree conflicts from being detected,
> because it hides situations where an update would result in a 'delete vs.
> delete' conflict. (I believe Subversion <= 1.4 used to allow this.)
> Similarly, I don't think we want to auto-merge if a directory is locally
> deleted and a newer version has property changes in the repository.
> Rather, we want the delete vs property change to conflict in the wc.

Stefan proposes to allow the commit of a propchange on a directory whose
revision changed only due to bubble-up, and whose children only changed
due to (prop or text) modifications; i.e., no additions, deletions,
moves or replacements of children, or the directory itself, happened in
the meantime.

That we do not allow prop changes on a directory with these constraints
has been known and recognized as an unnecessary limitation for a long
time (since before 1.0, in fact), but no-one has yet had the motivation
to change the txn-rebase (a.k.a. merge) code. Given that repositories
these days are getting really huge, and commits of merges take longer,
the motivation is now clear. Note that merge gets the most benefit from
this enhancement, since you can't avoid a propchange.

-- Brane

Branko Čibej | Director of Subversion
WANdisco // Non-Stop Data
e. brane_at_wandisco.com
Received on 2014-01-15 13:27:59 CET

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.