On 15/08/2011, at 22:54 , Stefan Sperling stsp-at-elego.de |subversion users list| wrote:
> On Mon, Aug 15, 2011 at 01:34:15AM +0000, Simon wrote:
>> We have a main master repository and a number of mirror slave
>> repositories at a bunch of locations that are set up as webdav
>> transparent write-through proxies. These are synced by a process
>> similar to svnsync, and this all seems to work okay.
>
> So you're using a different synchronisation process than svnsync?
> How does it work? What prevents you from using svnsync? What's the
> commit-transfer latency of your procedure compared to the latency of svnsync?
Our sync mechanism essentially uses svnadmin dump at the master and svnadmin load at the slaves with some home cooked supervisory functionality to gel it all together. This may not be optimal but my understanding is that our sync mechanism was developed to work around some prior problems in svnsync for large commits. It is possible that the original svnsync problem has long since been fixed but the infrastructure is now somewhat standardised and I have very limited control over the infrastructure that I have available.
>> However, it is inevitable that there is delay in the commits at the
>> master repository propagating out to the slaves. This is not usually a
>> problem, except when a large commit has been made where the transfer
>> time of the revisions data is significant. In this situation the a
>> client that uses the slave repository can have its commit blocked
>> because it is unable to update to the latest revision because the
>> slave repository is out of sync. This is unfortunate because it makes
>> the slave repository somewhat useless until the sync has time to
>> resolve itself. In a recent situation our slave was out of sync for
>> around 3.5 hours.
>>
>> Is there a workaround for this situation?
>
> You're not telling why one or more commits took 3.5 hours to sync.
> Why was the date set larger than usual? Maybe if you provide more
> information about this someone will come up with a workaround.
This was a 700Mb checkin synced out to slaves over a loaded WAN link, so this was predominately transfer time of the data blob for the commit.
> One scenario where a large commit can happen is when a lot of new
> revisions are imported from a dumpfile, e.g. when a project is moved
> from one repository to another.
> The Apache Software Foundation (ASF) uses a write-through proxy setup.
> The main server is in the US and the mirror is in Europe. Occasionally,
> new code is imported with history when a project joins the ASF.
> The new revisions are stored in a dump file which is copied to the master
> and the slave. Next, commit access is temporarily disabled on both servers.
> The dump file is loaded into both repositories. svnsync meta data is updated
> on the slave to mark the current head revision as synced (this number
> is in the svn:sync-last-merged-rev revision property at revision 0).
> Now when commit access is re-enabled the master and slave are already
> in sync. The resulting downtime is lower than if the newly imported
> revisions were imported at the master and synced via svnsync.
Received on 2011-08-16 15:03:52 CEST