[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Parallelising SVN Checkout

From: Mark Phippard <markphip_at_gmail.com>
Date: Tue, 4 Sep 2012 10:30:49 -0400

On Tue, Sep 4, 2012 at 10:26 AM, Jonathan Holloway <
jonathan.holloway_at_gmail.com> wrote:

> Has anybody had any success in parallelising Subversion checkouts in the
> past on a subfolder level
> to improve performance at all?
> By this I mean using svn sparse directories with --depth intermediates to
> produce a skeleton structure,
> such as:
> project
> * subfoldera - handled by a checkout thread
> * subfolderb - handled by a checkout thread
> http://svnbook.red-bean.com/en/1.5/svn.advanced.sparsedirs.html
> then forking a process/starting a thread to svn update the subfolders?
> Does this make sense from a
> performance point of view or is the bottleneck of disk I/O always
> hit pretty early on by doing this?
> http://stackoverflow.com/questions/4160070/can-an-svn-checkout-be-multi-threaded
> I've done this with a Python script and the multiprocessing module (and
> daemon processes) so far. I
> just wanted to check to see if there was an existing solution to this.

You will not be able to do this with SVN 1.7+ as there is a single working
copy admin area that will be locked by the first process that obtains the
lock. If you are using HTTP(S), you can switch to using the ra_serf
library which opens multiple connections to the server and fetches multiple
files at once. In 1.8, this will be the new HTTP client library.

Mark Phippard
Received on 2012-09-04 16:31:21 CEST

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.