[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Corrupt repository on large commit (was Re: svn+ssh on Win32)

From: Russell Glaue <rglaue_at_cait.org>
Date: 2003-07-21 16:51:20 CEST

We got errors similar to this too.
We are checking in 2000 files or more and it croaks close to the end.
We ended up removing the repository and doing the check-in again
(crossing our fingers).
One repos with well over 3000 files always croaked and we had to split
the check-in to 3 phases.
A second repository was about 2000 files and it croaked the first time
and second time. The third time it made it all the way through without
croaking on us.
This was with Subversion server 0.17 though.

If you have 0.25, try the suggestions discussed here (upgrade to 0.25
if you do not have it).
set local config 'http-compression = no'
set local config 'http-timeout = 7200'

We have only did the later, and things seem a little better. We will
try turning compression off if problems arise again.

For those who are interested in the time for check-in. The above two
repositories took about 2 hours plus to complete the check-in. This is
only what I remember.
-RG

On Sunday, Jul 20, 2003, at 23:27 America/Chicago, Steve Williams wrote:

>>> As for the checkout -- "chunk read error"? Never even heard of that
>>> error... from anyone, actually, ever. I'd like to hear more about
>>> it.
>>
>> I'll copy and paste the exact text of the error when I get to the
>> office tomorrow.
>
> After checking out approximately half of the repository, I got this...
>
> svn: RA layer request failed
> svn: REPORT request failed on '/repos/ty2/!svn/vcc/default'
> svn: REPORT of '/repos/ty2/!svn/vcc/default': Error reading chunked
> response
> body (http://julius)
>
> I did a recover on the repository, tried the checkout again and got
> the same
> result.
>
> The directory tree was 2.07GB of 15,793 files in 102 directories. I
> had
> committed the majority of the tree in small sections (very time
> consuming)
> but it was towards the end when one directory tree of 285MB in 9,761
> files
> in 44 directories failed with a MERGE timeout. This resulted in the
> corrupted repository described above.
>
> No mod_deflate in httpd. No mention of 'deflate' in httpd.conf.
> http-compression set to "no" in global section of
> %APPDATA%\Subversion\servers
> Add and commit a large (2.07GB, 15,793 files, 102 directories)
> directory
> tree.
>
> I am trying again with a fresh repository.
>
> The Data directory contains all 2.07GB of files to be added to the
> repository.
>
> F:\test>svn add Data
> F:\test>svn commit Data -m "Initial commit"
>
> After almost two hours of watching dots go past, it got to a point
> where the
> dots stopped, then there was a huge amount of network traffic followed
> by a
> huge amount of drive access on the client side. So much drive
> activity that
> it slowed the client machine down to a crawl. It was after several
> minutes
> of this client side drive activity that the MERGE timeout occurred.
>
> ....................svn: RA layer request failed
> svn: Commit failed (details follow):
> svn: MERGE request failed on '/repos/test/trunk/data'
> svn: MERGE of '/repos/test/trunk/data': timed out waiting for server
> (http://julius)
>
> Delete F:\test and try a checkout.
>
> F:\>svn co http://julius/repos/test/trunk test
>
> Checks out thousands of files and then comes up with...
>
> svn: RA layer request failed
> svn: REPORT request failed on '/repos/test/!svn/vcc/default'
> svn: REPORT of '/repos/test/!svn/vcc/default': Error reading chunked
> response body (http://julius)
>
> Trying the same thing with svn: protocol now and it seems to have hit
> a wall
> at a few thousand files into the commit. There is very little disk
> activity
> on the client side, constant disk activity on the server side (the HDD
> light
> is full on and I'm the only person using it) and regular network
> activity,
> but there has been no more output from the client. It's been like
> that for
> almost an hour now. I don't know what it is doing. I cannot
> communicate
> with the server. An existing ssh session is responding extremely
> slowly
> (one character echoed every ten to twenty seconds, if that). The
> P3-733
> server has been brought to its knees. Just wondering if it has got
> itself
> into an endless loop of sorts?
>
> Sly
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Mon Jul 21 16:52:08 2003

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.