[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: crash managing a large FSFS repository

From: Josh Pieper <jjp_at_pobox.com>
Date: 2004-12-13 15:21:11 CET

Chase Phillips wrote:
> As a follow-up to my thread "svn resource requirements with large
> repositories?" (http://svn.haxx.se/users/archive-2004-11/0180.shtml), I
> was recently able to try out the same procedure with revision 12289 from
> Subversion's trunk. With this revision I experience the same resource
> usage issues that led me to raise this issue at first.
>
> As a refresher, our project is developing a software image that runs on
> top of NetBSD. We need to store NetBSD in a revision-controlled
> environment to track the changes we make to the operating system and
> kernel. I decided to create this new repository locally on the disk in
> FSFS format (our current repo that stores application source is in BDB
> format).
>
> After importing the NetBSD source code and then copying it onto a branch,
> a subsequent checkout leads to a core dump. I've attached one of the
> stack traces from my two attempts to this email (each attempt takes
> upwards of 15 minutes before svn dumps core). The second stack trace
> differs from the first only in memory addresses of variables, though it
> can be sent as well if needed.

I just attempted this experiment using a local FSFS repository and
what I think is the entirety of the NetBSD source tree. Import was
successful after 42 minutes wall time. Maximum memory usage during
the majority of the import was 19M, however, after the final revision
file was moved into place, memory usage spiked to 44M. It could just
be the recursive delete of the completed transaction directory. It
had around 120,000 files in it after finalization.

Checkout was successful too, after 43 minutes wall time. Memory usage
climbed steadily throughout the entire checkout, peaking at 85M. I
didn't remember that there was a component of checkout/update that was
supposed to be linear in memory?

Just for record, the NetBSD sources I used totaled 973 megabytes, with
95,900 files or so. The checked out working copy totaled 3,336
megabytes in size. The experiments were on my relatively unloaded
Athlon XP 1900 with a 5400 rpm 80G hard drive.

-Josh

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Mon Dec 13 15:25:21 2004

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.