crash managing a large FSFS repository

As a follow-up to my thread "svn resource requirements with large
repositories?" (http://svn.haxx.se/users/archive-2004-11/0180.shtml), I
was recently able to try out the same procedure with revision 12289 from
Subversion's trunk. With this revision I experience the same resource
usage issues that led me to raise this issue at first.

As a refresher, our project is developing a software image that runs on
top of NetBSD. We need to store NetBSD in a revision-controlled
environment to track the changes we make to the operating system and
kernel. I decided to create this new repository locally on the disk in
FSFS format (our current repo that stores application source is in BDB
format).

After importing the NetBSD source code and then copying it onto a branch,
a subsequent checkout leads to a core dump. I've attached one of the
stack traces from my two attempts to this email (each attempt takes
upwards of 15 minutes before svn dumps core). The second stack trace
differs from the first only in memory addresses of variables, though it
can be sent as well if needed.

The Subversion issue tracker holds 4 issues that come close to addressing
this but for one reason or another don't match up well enough to allow me
to assume they should be used as the target for this issue.

Issue 602 - http://subversion.tigris.org/issues/show_bug.cgi?id=602

  "import of large trees can bloat memory on client side"

  The last comment to this bug was made 2002/11 and it appears purposed to
  handle import efficiency (I've not had this resource issue doing an
  import of the source code).

Issue 1702 - http://subversion.tigris.org/issues/show_bug.cgi?id=1702

  "Initial checkout should scale well for large projects"

  This issue focuses on checking out a revision from a remote repository.
  In my scenario, I am checking out from a local repository.

Issue 2067 - http://subversion.tigris.org/issues/show_bug.cgi?id=2067

  "Perf issues with BDB and directories with a large number of items"

  This bug mentions similar problems with FSFS, though the bug summary
  refers strictly to BDB. Is it meant to cover only issues with BDB-based
  repositories?

Issue 2137 - http://subversion.tigris.org/issues/show_bug.cgi?id=2137

  "svn update" extremely slow with large repository and FSFS"

  The common problems I experience are with checkouts and commits back to
  the local repository. Again, I'm using a late revision of the trunk
  (r12289).

Should one of the above issues be used for tracking this problem? Or
should I file a new issue, presuming I'm running into a bug in the source
and not some problem local to my system? Any suggestions for what to try
next?

PS Initial work on this issue was done by Eric Gillespie. He applied one
patch to the trunk at r11701 and another at r11706. The appropriate dev
threads can be found at:

  "[PATCH] Fix svn_io_remove_dir pool usage", 2004/11/01
  http://svn.haxx.se/dev/archive-2004-11/0000.shtml

  "[PATCH] Fix fsfs finalization memory usage", 2004/11/01
  http://svn.haxx.se/dev/archive-2004-11/0001.shtml

Thanks,
Chase
CUWiN project

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org