[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: numerous small 512 byte working copy reads and writes

From: <kmradke_at_rockwellcollins.com>
Date: Wed, 9 Jan 2008 14:42:18 -0600

"Erik Huelsmann" <ehuels_at_gmail.com> wrote on 12/18/2007 05:05:01 AM:
> On 12/18/07, Norbert Unterberg <nunterberg_at_gmail.com> wrote:
> > On Dec 17, 2007 9:07 PM, <kmradke_at_rockwellcollins.com> wrote:
> > > "Erik Huelsmann" <ehuels_at_gmail.com> wrote on 12/17/2007 04:59:58 AM:
> >
> > > > The buffer size comes from BUFSIZ which is used to copy files in
APR.
> > > > They probably use BUFSIZ because the definition of the constant is
> > > > that it should return a buffer size for efficiently doing file IO.
> > > >
> > > > Probably the CRT he's using is defining BUFSIZ to 512... (Which I
> > > > agree, is quite small...)
> > >
> > > Thanks for the pointers. I was wondering if it could be an APR
issue.
> > >
> > > I haven't yet verified, but I assume the windows command line client
is
> > > compiled with visual studio, which would use the standard Microsoft
C
> > > runtime.
> >
> > You are on the wrong track here.The problem is not what BUFSIZ is
> > defined on the different platforms but how APR uses it.
> > BUFSIZ is the size of the buffer that can be set for buffered streams
> > with the setbuf() call. Nothing more. MSDN just says: "BUFSIZ is the
> > required user-allocated buffer for the setvbuf routine." BUFSIZ is NOT
> > a suggestion for a buffer size when dealing with large binary files.
> > So the bug is in APR where BUFSIZ is used for something it was not
> designed for.
> >
> > This has already been discussed for years on this list, but
> nothing has changed:
> > http://subversion.tigris.org/servlets/ReadMsg?listName=dev&msgNo=82087
> >
http://subversion.tigris.org/servlets/ReadMsg?listName=dev&msgNo=113549
> >
> > So it is clearly an apr and not a windows issue.
>
> Providing a patch might help: They define a buffer size of their own
> (APR_FILE_BUFSIZE, defined to 4096 on Win32) which should probably be
> used for binary file copies since it's used for buffered files as
> well.
>
>
> Though 16k seemed to perform best on Win32 a year or 2 ago, when
> tested by an svn dev; while no negative impact was detected on Unix
> systems when shrinking from larger sizes to this relatively small
> buffer size. All this was done on local filesystems.

I submitted a bug report to APR to increase the size of the buffer size.
http://issues.apache.org/bugzilla/show_bug.cgi?id=44193

Kevin R.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe_at_subversion.tigris.org
For additional commands, e-mail: dev-help_at_subversion.tigris.org
Received on 2008-01-09 21:45:19 CET

This is an archived mail posted to the Subversion Dev mailing list.