[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Corrupt repository on large commit (was Re: svn+ssh on Win32)

From: Russell Glaue <rglaue_at_cait.org>
Date: 2003-07-21 17:10:37 CEST

number corrections.
        Project #1: 13,000 files, 789 MB
        Project #2: 5,000 files, 892 MB

Project #1 takes much longer (longer than Project #2) to perform the
initial checkout.
Again, it is my belief that it is the number of files, not the size of
them, that causes the checkout to take so long.
-RG

On Monday, Jul 21, 2003, at 09:51 America/Chicago, Russell Glaue wrote:

> We got errors similar to this too.
> We are checking in 2000 files or more and it croaks close to the end.
> We ended up removing the repository and doing the check-in again
> (crossing our fingers).
> One repos with well over 3000 files always croaked and we had to split
> the check-in to 3 phases.
> A second repository was about 2000 files and it croaked the first time
> and second time. The third time it made it all the way through without
> croaking on us.
> This was with Subversion server 0.17 though.
>
> If you have 0.25, try the suggestions discussed here (upgrade to 0.25
> if you do not have it).
> set local config 'http-compression = no'
> set local config 'http-timeout = 7200'
>
> We have only did the later, and things seem a little better. We will
> try turning compression off if problems arise again.
>
> For those who are interested in the time for check-in. The above two
> repositories took about 2 hours plus to complete the check-in. This is
> only what I remember.
> -RG
>
>
> On Sunday, Jul 20, 2003, at 23:27 America/Chicago, Steve Williams
> wrote:
>
>>>> As for the checkout -- "chunk read error"? Never even heard of that
>>>> error... from anyone, actually, ever. I'd like to hear more about
>>>> it.
>>>
>>> I'll copy and paste the exact text of the error when I get to the
>>> office tomorrow.
>>
>> After checking out approximately half of the repository, I got this...
>>
>> svn: RA layer request failed
>> svn: REPORT request failed on '/repos/ty2/!svn/vcc/default'
>> svn: REPORT of '/repos/ty2/!svn/vcc/default': Error reading chunked
>> response
>> body (http://julius)
>>
>> I did a recover on the repository, tried the checkout again and got
>> the same
>> result.
>>
>> The directory tree was 2.07GB of 15,793 files in 102 directories. I
>> had
>> committed the majority of the tree in small sections (very time
>> consuming)
>> but it was towards the end when one directory tree of 285MB in 9,761
>> files
>> in 44 directories failed with a MERGE timeout. This resulted in the
>> corrupted repository described above.
>>
>> No mod_deflate in httpd. No mention of 'deflate' in httpd.conf.
>> http-compression set to "no" in global section of
>> %APPDATA%\Subversion\servers
>> Add and commit a large (2.07GB, 15,793 files, 102 directories)
>> directory
>> tree.
>>
>> I am trying again with a fresh repository.
>>
>> The Data directory contains all 2.07GB of files to be added to the
>> repository.
>>
>> F:\test>svn add Data
>> F:\test>svn commit Data -m "Initial commit"
>>
>> After almost two hours of watching dots go past, it got to a point
>> where the
>> dots stopped, then there was a huge amount of network traffic
>> followed by a
>> huge amount of drive access on the client side. So much drive
>> activity that
>> it slowed the client machine down to a crawl. It was after several
>> minutes
>> of this client side drive activity that the MERGE timeout occurred.
>>
>> ....................svn: RA layer request failed
>> svn: Commit failed (details follow):
>> svn: MERGE request failed on '/repos/test/trunk/data'
>> svn: MERGE of '/repos/test/trunk/data': timed out waiting for server
>> (http://julius)
>>
>> Delete F:\test and try a checkout.
>>
>> F:\>svn co http://julius/repos/test/trunk test
>>
>> Checks out thousands of files and then comes up with...
>>
>> svn: RA layer request failed
>> svn: REPORT request failed on '/repos/test/!svn/vcc/default'
>> svn: REPORT of '/repos/test/!svn/vcc/default': Error reading chunked
>> response body (http://julius)
>>
>> Trying the same thing with svn: protocol now and it seems to have hit
>> a wall
>> at a few thousand files into the commit. There is very little disk
>> activity
>> on the client side, constant disk activity on the server side (the
>> HDD light
>> is full on and I'm the only person using it) and regular network
>> activity,
>> but there has been no more output from the client. It's been like
>> that for
>> almost an hour now. I don't know what it is doing. I cannot
>> communicate
>> with the server. An existing ssh session is responding extremely
>> slowly
>> (one character echoed every ten to twenty seconds, if that). The
>> P3-733
>> server has been brought to its knees. Just wondering if it has got
>> itself
>> into an endless loop of sorts?
>>
>> Sly
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
>> For additional commands, e-mail: dev-help@subversion.tigris.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Mon Jul 21 17:11:30 2003

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.