[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: svnsync crashes on a huge commit

From: Stefan Sperling <stsp_at_elego.de>
Date: Thu, 11 Jul 2013 12:20:40 +0200

On Thu, Jul 11, 2013 at 02:04:11PM +0400, Anatoly Zapadinsky wrote:
> I've increased swap size to 8gb, increased opened files handles limit to
> 200k and ran the process at night when the server is not very busy.
> Finally svnsync form a revision copy in db, but hang after it. It didn't
> print "Committed revision" just hanged on a set of dots after "Transmitting
> file data....". I've killed it and ran "svnadmin verify". The revision copy
> seems to be fine. And it was successfully packed by svnadmin pack at the
> end of mirroring process.
> svnsync is a buggy crap! I guarantee this. And no one cares. No one going
> to opened an issue.

This seems to be a problem in serf when it tries to restart a request.

You're running into this code path, which shouldn't be happening:

      status = serf_bucket_read(response, PARSE_CHUNK_SIZE, &data, &len);

      if (SERF_BUCKET_READ_ERROR(status))
          return svn_ra_serf__wrap_err(status, NULL);

      ctx->read_size += len;

      if (ctx->skip_size)
          /* Handle restarted requests correctly: Skip what we already read */
          apr_size_t skip;

          if (ctx->skip_size >= ctx->read_size)
            /* Eek. What did the file shrink or something? */
              if (APR_STATUS_IS_EOF(status))
line 1649 ---> SVN_ERR_MALFUNCTION();

So please file an issue. This is clearly a bug.

It looks like you're using Subversion 1.8.
If you're not using Subversion 1.8, then you shouldn't be using
serf unless you tweak the client configuration.

In the mean time, please check if Subversion 1.7 with neon works.

> On Tue, Jul 9, 2013 at 3:30 PM, Anatoly Zapadinsky <zapadinsky_at_gmail.com>wrote:
> > svnsync failed to sync repository with a single commit containing 56000
> > files. This commit handled perfectly by console svn and tortoise svn. We
> > can checkout this folder structure and so on. But when I tried to sync this
> > commit to the local mirror repository it fails.
> > 32 bit version of svnsync crashed with out of memory exception.
> > 64 bit version made a transaction folder containing 117000 files in it,
> > allocated like 3gb of memory and finally crashed with this diagnostic:
> > "svn: E235000: In file 'subversion/libsvn_ra_serf/util.c' line 1649:
> > internal malfunction".
> > May be this commit is not an example of the best practice but it crash
> > only svnsync all the other svn tools handle it perfectly.
> > What should I do? How to mirror this repository? Is it a bug or not?
> >
Received on 2013-07-11 12:21:18 CEST

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.