[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: svnsync will not sync 527Mb revision

From: David Ferguson <ferguson.david_at_gmail.com>
Date: Thu, 19 Nov 2009 12:02:07 -0500

Philip,

We had to abandon svnsync due to the problem that you encountered. We got
lots of "Chunk delimiter invalid" messages during sync.

We instead use svnadmin dump --incremental and svnadmin load to copy over
the data every time a commit is made. Not very elegant but it works 100% of
the time regardless of the file size.

I like Mark's idea to sync via svn:// though. The http:// protocol seems to
be a completed dog for this replication task. I'll have to try that out.

David

On Wed, Nov 18, 2009 at 11:18 AM, <kmradke_at_rockwellcollins.com> wrote:

>
> Pat Farrell <pfarrell_at_pfarrell.com> wrote on 11/17/2009 10:23:40 PM:
>
> > Ryan Schmidt wrote:
> > > On Nov 17, 2009, at 17:27, Pat Farrell wrote:
> > >> I'm pretty sure that this is considered a "doctor, doctor" problem.
> > >>
> > > That's probably not a reasonable attitude to take for a tool like
> > > svnsync that's supposed to be able to replicate existing
> > > repositories. He said he already committed the large revision. We all
> > > know there is no svnadmin obliterate and that it's difficult to
> > > remove revisions from a repository later. And some users have valid
> > > reasons for wanting to store large binary files in a repository.
> > > (Video game development, for example, would involve large files for
> > > textures, audio and video.)
> >
> > OK, I'll grant that, but in general, my experience is that SVN is not a
> > great tool for large binary files. I've not worked with the specific,
> > "svnsync" but in general, big files tend to clog up the whole process.
> >
> > I agree that if its supported, it should work.
> >
>
> > I suggest that the svn developers consider strongly warning people that
> > using it for big binary things is not optimal. It works well for source
> > code, and at least "OK" for small binary things (couple of pages of Word
> > or OpenOffice documents). But when the files get into hundreds of
> > megabytes, for each revision, I think SVN is not the right tool.
>
> FWIW, our largest single revision is >20G... We don't use svnsync, but you
> are probably running into a network timeout where the client is taking
> too long writing to disk causing the network connection to timeout before
> it is able to request the next chunk from the server. We had to increase
> timeout values when using http:// when using large working copies
> with slow disk i/o on the client. Not sure where to tweak on svn://...
>
> Kevin R.
>

------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=2420110

To unsubscribe from this discussion, e-mail: [users-unsubscribe_at_subversion.tigris.org].
Received on 2009-11-19 18:02:53 CET

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.