[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

RE: svn commit: r1344347 - /subversion/trunk/subversion/libsvn_subr/config_file.c

From: Bert Huijben <bert_at_qqmail.nl>
Date: Thu, 31 May 2012 09:59:47 +0200

> -----Original Message-----
> From: Greg Stein [mailto:gstein_at_gmail.com]
> Sent: donderdag 31 mei 2012 9:49
> To: Ivan Zhakov
> Cc: dev_at_subversion.apache.org; Bert Huijben
> Subject: Re: svn commit: r1344347 -
> /subversion/trunk/subversion/libsvn_subr/config_file.c
>
> On Thu, May 31, 2012 at 3:41 AM, Ivan Zhakov <ivan_at_visualsvn.com>
> wrote:
> > On Wed, May 30, 2012 at 8:52 PM,  <rhuijben_at_apache.org> wrote:
> >> Author: rhuijben
> >> Date: Wed May 30 16:52:36 2012
> >> New Revision: 1344347
> >>
> >> URL: http://svn.apache.org/viewvc?rev=1344347&view=rev
> >> Log:
> >> Remove about 15% of the cpu overhead of svnserve and httpd while
> running the
> >> test suite by introducing a very simple parser buffer in the config
file
> parser
> >> code.
> >>
> >> The nice clean stream translation code had an 7 times overhead on this
> very
> >> performance critical code path, while clients of svnserve and httpd
were
> >> waiting. (Before this patch the this code path used 22% of the cpu
time,
> >> now just 3%)
> >
> > Great improvement, but it may be worth to factor out this code to
> > special stream like svn_stream_buffered().
>
> When I saw this change, my first thought was, "wtf? just read the
> whole damned file into memory". (I just haven't had time to look into
> this to provide some actual advice beyond amazement)
>
> Config files are NOT 100 megabytes. Read the damned things into memory
> in one giant piece. I/O is reduced, memory is available. Everybody
> wins.

I'm sure it could use further improvements. I don't know what really large
authz files some users have, but reading in 10 Kbyte chunks as it does now
gives a huge boost by itself.

The current code assumes the line endings have at least a '\n' (and ignores
'\r'), which would work for any typical user, but we used to support '\r' by
itself, because that is what our translated streams do.
I'm not sure if we should call this a regression as we probably never really
intended to support that. (And the Mac switched to '\n' years ago)

Most of the server side time was spend in reading the FSFS config files. I
didn't know we had that many user editable files in a typical repository and
all those commented lines explaining the flags were (and are) the
performance killer.

        Bert
Received on 2012-05-31 10:00:25 CEST

This is an archived mail posted to the Subversion Dev mailing list.