[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: svn commit: r1586391 - in /subversion/branches/thunder: BRANCH-README notes/thundering-herd.txt

From: Stefan Fuhrmann <stefan.fuhrmann_at_wandisco.com>
Date: Sun, 20 Apr 2014 21:28:51 +0200

On Mon, Apr 14, 2014 at 6:17 PM, Ivan Zhakov <ivan_at_visualsvn.com> wrote:

> On 12 April 2014 00:01, Stefan Fuhrmann <stefan.fuhrmann_at_wandisco.com>
> wrote:
> > It seems that the document in notes/ did not make it clear what
> > the actual problem is and how it applies to Subversion servers.
> > Let me try to illustrate it.
> >
> [...]
> >
> > Should these trends be confirmed for "real" HW, networks and ra_serf,
> > I'd love to see this code in 1.9 - after due review and feedback.
> >
> Hi Stefan,
> I got the idea from original description, but thank you for explaining
> it one more time. Anyway my concerns are still valid and I am -1 on
> implementing this at FSFS layer. Especially if we're going to release
> Subversion 1.9 in 2014.

Having taken more measurements and analysis on
my server hardware last weekend, it seams that
mod_dav_svn requires a refined throttling scheme.

So, unless 1.9 gets massively delayed, I now schedule
the thunder branch merger for 1.10.

> I'm also skeptical on FSFS full texts caching. I think that caching
> worth only for small metadata, that doesn't consume a lot of memory
> but expensive to build. But that's different story.

[Only to give you a data point:]

Well, it can be disabled and it can even be moved out
to memcached (and is the only the item type where a
network indirection is acceptable).

However, constructing fulltexts from data that is not
in disc cache can be very expensive: we read 10-ish
concurrent data streams and then do quite a bit of
unzipping and checksumming on the data.

Apache is already needs 8 CPU cores to saturate a
single 10Gb network connection (uncompressed, non-
encrypted source code) *with* 100% fulltext cache hits.
Adding more CPU load can be a problem.

-- Stefan^2.
Received on 2014-04-20 21:29:23 CEST

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.