Julian,
Here's a simple, reproducible test case:
1. download a file 150Mb or larger. I used a copy of the latest Ubuntu ISO for this test.
2. create an svn 1.6.6 repository under Apache 2.2.3 using the following subversion.conf file (make sure apache owns everything and you've configured your AuthUserFile correctly):
LoadModule dav_svn_module modules/mod_dav_svn.so
LoadModule authz_svn_module modules/mod_authz_svn.so
<Location /svn>
DAV svn
SVNPath /var/www/svn
SSLRequireSSL
AuthType Basic
AuthName "SVN"
AuthUserFile /var/www/svn_auth.dat
SVNPathAuthz off
Require valid-user
</Location>
3. Check in a simple one line text file as your first revision; this is so you can tell at least a normal revision syncs.
4. Check in the large file as your second revision.
5. Check in another one line text file as your third revision; this is to see if your sync completed without having to watch
Verify everything checked in OK. I use tortoise for checking in and browsing.
Create an empty repository as per the redbook; I add "SSLProxyEngine on" and an SVNMasterURI to make my slave nice.
Do the sync. It'll fail with one of several messages during the sync of the second revision.
My production repository contains terrain files, ESRI shape files, datasets, etc used in the project; the first 600 or so revisions are just software source and replicate fine; the trouble starts when the larger datasets are first introduced.
I run under RHEL5 with http/svn from the dag version of rpmfind. The main server has eight processors and 64GB ram; only other app right now is trac. The slave server is similar.
Dan
-----Original Message-----
From: Julian Foad [mailto:julian.foad_at_wandisco.com]
Sent: Tuesday, November 10, 2009 11:28 AM
To: Bitts, Dan
Cc: dev_at_subversion.tigris.org
Subject: Re: svnsync large files
On Thu, 2009-10-29, Bitts, Dan wrote:
> I’m running svnsync on RHEL5 svn version 1.6.5 CollabNet binaries
> syncing to a remote RHEL5 version 1.4.5 CollabNet binaries repository.
> If the source repository contains a very large binary file (roughly
> 500Mb) the sync will crash with varying messages (usually source
> timeout related).
It doesn't sound like a memory consumption problem, but the only thing
in the issue tracker that sounds at all similar is "svnsync runs out of
memory syncing mythtv repository"
<http://subversion.tigris.org/issues/show_bug.cgi?id=3478>. Could you
watch the memory consumption and see if your issue looks at all similar
to this issue?
And please show us the exact output when it does "crash", several
examples if they're varying.
Can you try to narrow it down to a reproducible test case, perhaps by
creating a test repository that contains only a check-in of your big
file and syncing that?
- Julian
> Using svnadmin dump, svndumpfilter exclude, svnadmin load to filter
> out the very large file the repository will sync. The source
> repository is roughly 4Gb and has just under 900 revs; the very large
> file was added around rev 800. We have other more reasonable binaries
> in the source that seem to go across just fine. We added and
> committed the file using tortoise on XP without any problems, and
> check it out frequently with a variety of clients.
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=2425682
Please start new threads on the <dev_at_subversion.apache.org> mailing list.
To subscribe to the new list, send an empty e-mail to <dev-subscribe_at_subversion.apache.org>.
Received on 2009-12-01 11:47:29 CET