[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Subversion and Very,Very Large Repositories

From: Mark Phippard <markp_at_softlanding.com>
Date: 2006-02-28 16:42:27 CET

"Roger Ashby" <roger.ashby@gmail.com> wrote on 02/28/2006 10:39:23 AM:

> On 2/28/06, Mark Phippard <markp@softlanding.com> wrote:
>
> > I think the main issue is that when you commit something to one of
those
> > folders, the resulting revision file has to contain meta-information
about
> > all of the files in that folder so the revision files are just larger
than
> > would be ideal. In your case, that information probably pales in
> > comparison to what you are committing, but when it is a text file it
turns
> > what would be a 1K revision file into a 100K revision file.
>
> So if I understand you correctly if I have a directory with 10,000
> image files, and I add 10 new files the resulting revision would be
> the size of the entire directory and not just the delta: which in this
> particular case would be just the newly added files?

Be sure to use Reply to All so that the mailing list is cc:ed

It would not contain the contents of any of those files, just some meta
information such as the filenames and the last revision they were changed
in. The point is that this really adds up when you multiply it by 10,000.

> > The other issue is that Subversion does not let you checkout files,
just
> > folders. So when you checkout these folders you have to always
checkout
> > all of those thousands of images.
>
> Checkouts should be rare. The reason that I choose SVK is because
> these files need to be always available in a Read Only format to our
> internal process and users. I set up a skeleton directory structure
> for the users to added new files and file changes to. I wrote a
> script that iterates over the skeleton directory and moves the changes
> into the Read Only area (after a fair bit of file integrity checking)
> which is really a SVK sandbox that will never be removed. The script
> then adds new files that weren't there before, and overwrites old
> files with new changes it grabbed from the skeleton directory and
> commits the whole repository. That was the plan anyway. :)

You might also consider using the WebDAV feature. Client could use the
WebDAV facilities of their OS to treat the repository like a network
volume.

Mark

_____________________________________________________________________________
Scanned for SoftLanding Systems, Inc. and SoftLanding Europe Plc by IBM Email Security Management Services powered by MessageLabs.
_____________________________________________________________________________

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Tue Feb 28 18:02:15 2006

This is an archived mail posted to the Subversion Users mailing list.