[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Directory size problem?

From: Hal Mahaffey <hmahaffey_at_aol.com>
Date: 2006-03-30 23:24:52 CEST
First, I apologize if this has been asked many times, but I can't find where the archive of postings for this list are.  It might be nice to add a "Search the Archives" link after the "Unsubscribe" link in the postings.  :)  I did find the FAQ and did not find my question there.

I'm new to Subversion, as will soon be obvious.  :)  While reading "the Free Book", I came upon a comment where Berkely DB and FSFS are compared that struck home with me. Under "Scalability: number of revision trees", a comment is made about FSFS: "some older native filesystems don't scale well with thousands of entries in a single direrectory".

I didn't know how big a deal that was until I saw that every revision changeset to a repostiory is maintained in a single file, and that a single directory contains *all* of these files.  With a large repository, I can see repository revisions reaching several tens of thousands (we have quite a lot of code).  We already had a case where a single directory contained build manifests for 15,000 builds.  Performance accessing these files (in CVS) was noticeably painful.   We solved it by indexing the files in subdirs based on the 1st character of the build name: A...Za...z.

Has anyone noticed performance problems with repositories with 10,000+ revisions?

What "older native filesystems" have this problem?  We were using Veritas v3.5 on EMC Symms and noticed the problem.  Are there any recommended filesystems?

If there *is* a problem with larger repositories, has the Subversion community considered indexing the 'db/revs' directory with, say, the first digit (so revision 2423 would go in db/revs/2)?  Just a thought.

Thank you!


--------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org For additional commands, e-mail: users-help@subversion.tigris.org Received on Thu Mar 30 23:27:00 2006

This is an archived mail posted to the Subversion Users mailing list.