Hi,
I just tried svnsyncing the chromium repository to my local machine
(Subversion 1.8.0, standard SlikSvn binary) on Windows and the svnsync
process didn't get above 86 MB memory usage syncing the first 563 revisions.
This was a sync to a local harddisk.
The number of open handles is pretty stable around 135.
While typing the message the really huge r564 came through, but I don't see
any difference compared to the older number (Peak memory usage still below
87 MB).
Bert
From: Anatoly Zapadinsky [mailto:zapadinsky_at_gmail.com]
Sent: woensdag 17 juli 2013 14:04
To: Lieven Govaerts
Cc: Subversion Development
Subject: Re: svnsync crashes on a huge commit
I've run 64 bit version on virtual machine and access fsfs repository on
host windows computer through smb. It's because 32bit version of svnsync
crashed on host windows machine on this huge commit with out of memory
exception.
I've found an easy way to reproduce this excessive memory use in svnsync.
Try to mirror <https://src.chromium.org/chrome>
https://src.chromium.org/chrome They had done some very big commits in the
beginning. On my machine svnsync allocate more than a gigabyte of memory
syncing one of the first commits (in first 50). Please someone confirm that
you can reproduce this issue mirroring chromium repository.
On Wed, Jul 17, 2013 at 3:49 PM, Lieven Govaerts < <mailto:lgo_at_apache.org>
lgo_at_apache.org> wrote:
(bringing this to dev)
Devs,
Anatoly sent me some more info in this issue and log files of svnsync
1.8.0+serf 1.2.1 with logging enabled.
He's running svnsync from a https repository (ra_serf) to a local
repository (ra_local). This on an Ubuntu 12.08 VM with 64-bit binary.
In the log files I received doesn't abort but stops with this error
(after hours of syncing):
[2013-07-12T01:37:35.650011-07] [l: <http://192.168.222.132:53349>
192.168.222.132:53349
r: <http://10.14.3.25:443> 10.14.3.25:443] outgoing.c: cleanup - closed
socket, status 9
svnsync: E000005: Can't open file
'/media/windowsshare/db/transactions/391384-8e0c.txn/next-ids':
Input/output error
... (cleanup and end program)
As far as I can see in the logs in this particular run there's nothing
wrong on the receiving end in ra_serf, but there's a problem during
the commit action.
I suppose (to be confirmed) that his target repository is stored on an
nfs or smb share.
Svn has a retry feature in fsfs to retry opening files after receiving
an EIO error, see libsvn_fs_fs/fs_fs.c RECOVERABLE_RETRY_COUNT, but I
don't see this being used in read_next_ids (fs_fs.c) when opening the
next-ids file.
Can anyone more knowledgeable about fsfs confirm that this is a
possible explanation for this issue?
Lieven
On Tue, Jul 9, 2013 at 1:30 PM, Anatoly Zapadinsky <
<mailto:zapadinsky_at_gmail.com> zapadinsky_at_gmail.com> wrote:
> svnsync failed to sync repository with a single commit containing 56000
> files. This commit handled perfectly by console svn and tortoise svn. We
can
> checkout this folder structure and so on. But when I tried to sync this
> commit to the local mirror repository it fails.
> 32 bit version of svnsync crashed with out of memory exception.
> 64 bit version made a transaction folder containing 117000 files in it,
> allocated like 3gb of memory and finally crashed with this diagnostic:
"svn:
> E235000: In file 'subversion/libsvn_ra_serf/util.c' line 1649: internal
> malfunction".
> May be this commit is not an example of the best practice but it crash
only
> svnsync all the other svn tools handle it perfectly.
> What should I do? How to mirror this repository? Is it a bug or not?
Received on 2013-07-18 09:23:55 CEST