[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: large binary data repositories with many files

From: ListMan <listman_at_burble.net>
Date: 2006-04-12 02:13:44 CEST

sure, sorry for the garbled message

raw file size = 2.1G
# files(dirs) = 5331(2061)
unix copy = 6m
commit = 68mins
repos size = 1.2G
workArea size = 4.2G
# files(dirs) = 44011(22671)

the working area is pretty large, we have 30 users that will have the
a copy of the repos in their local working area. anyway we can reduce
this?

the commit is very slow, there's a strong possiblity that we'll have
30 users trying to do a commit at the same time when they get in for
work in the morning. How else can we reduce the commit time?

On Apr 11, 2006, at 4:10 PM, Nico Kadel-Garcia wrote:

> ListMan wrote:
>> hi, we have lots of binary data that we are trying to data manage
>> with subversion. as you can see from the example below svn can be
>> kind of slow, a commit can take up to 68mins.. are there certain
>> binary characteristics of files that lend themselves to efficient
>> diff generation? is there a place i can research the algorithm used
>> by svn? any ideas how we can speed this up with svn?
>>
>> raw
>> performance repository workarea
>> size files(dirs) copy add commit update
>> size size files(dirs)
>> 2.1G 5331(2061) 6m 4m 68mins
>> 1.2G 4.2G 44011(22671)
>
> Well, there are obvious ones: enough RAM on the server so that it
> doesn't have to swap when adding the binaries to the database, for
> example, might be very helpful.
>
> And unfortunately, the layout of your message was fairly scrambled:
> can you try a different format so it doesn't line wrap like this?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Wed Apr 12 02:15:25 2006

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.