I've run 64 bit version on virtual machine and access fsfs repository on
host windows computer through smb. It's because 32bit version of svnsync
crashed on host windows machine on this huge commit with out of memory
exception.
I've found an easy way to reproduce this excessive memory use in svnsync.
Try to mirror https://src.chromium.org/chrome They had done some very big
commits in the beginning. On my machine svnsync allocate more than
a gigabyte of memory syncing one of the first commits (in first 50). Please
someone confirm that you can reproduce this issue mirroring chromium
repository.
On Wed, Jul 17, 2013 at 3:49 PM, Lieven Govaerts <lgo_at_apache.org> wrote:
> (bringing this to dev)
>
> Devs,
>
> Anatoly sent me some more info in this issue and log files of svnsync
> 1.8.0+serf 1.2.1 with logging enabled.
>
> He's running svnsync from a https repository (ra_serf) to a local
> repository (ra_local). This on an Ubuntu 12.08 VM with 64-bit binary.
> In the log files I received doesn't abort but stops with this error
> (after hours of syncing):
>
> [2013-07-12T01:37:35.650011-07] [l:192.168.222.132:53349
> r:10.14.3.25:443] outgoing.c: cleanup - closed socket, status 9
> svnsync: E000005: Can't open file
> '/media/windowsshare/db/transactions/391384-8e0c.txn/next-ids':
> Input/output error
> ... (cleanup and end program)
>
> As far as I can see in the logs in this particular run there's nothing
> wrong on the receiving end in ra_serf, but there's a problem during
> the commit action.
>
> I suppose (to be confirmed) that his target repository is stored on an
> nfs or smb share.
>
> Svn has a retry feature in fsfs to retry opening files after receiving
> an EIO error, see libsvn_fs_fs/fs_fs.c RECOVERABLE_RETRY_COUNT, but I
> don't see this being used in read_next_ids (fs_fs.c) when opening the
> next-ids file.
>
> Can anyone more knowledgeable about fsfs confirm that this is a
> possible explanation for this issue?
>
> Lieven
>
> On Tue, Jul 9, 2013 at 1:30 PM, Anatoly Zapadinsky <zapadinsky_at_gmail.com>
> wrote:
> > svnsync failed to sync repository with a single commit containing 56000
> > files. This commit handled perfectly by console svn and tortoise svn. We
> can
> > checkout this folder structure and so on. But when I tried to sync this
> > commit to the local mirror repository it fails.
> > 32 bit version of svnsync crashed with out of memory exception.
> > 64 bit version made a transaction folder containing 117000 files in it,
> > allocated like 3gb of memory and finally crashed with this diagnostic:
> "svn:
> > E235000: In file 'subversion/libsvn_ra_serf/util.c' line 1649: internal
> > malfunction".
> > May be this commit is not an example of the best practice but it crash
> only
> > svnsync all the other svn tools handle it perfectly.
> > What should I do? How to mirror this repository? Is it a bug or not?
>
Received on 2013-07-17 23:44:06 CEST