[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: numerous small 512 byte working copy reads and writes

From: Erik Huelsmann <ehuels_at_gmail.com>
Date: 2007-12-18 12:05:01 CET

On 12/18/07, Norbert Unterberg <nunterberg@gmail.com> wrote:
> On Dec 17, 2007 9:07 PM, <kmradke@rockwellcollins.com> wrote:
> > "Erik Huelsmann" <ehuels@gmail.com> wrote on 12/17/2007 04:59:58 AM:
>
> > > The buffer size comes from BUFSIZ which is used to copy files in APR.
> > > They probably use BUFSIZ because the definition of the constant is
> > > that it should return a buffer size for efficiently doing file IO.
> > >
> > > Probably the CRT he's using is defining BUFSIZ to 512... (Which I
> > > agree, is quite small...)
> >
> > Thanks for the pointers. I was wondering if it could be an APR issue.
> >
> > I haven't yet verified, but I assume the windows command line client is
> > compiled with visual studio, which would use the standard Microsoft C
> > runtime.
>
> You are on the wrong track here.The problem is not what BUFSIZ is
> defined on the different platforms but how APR uses it.
> BUFSIZ is the size of the buffer that can be set for buffered streams
> with the setbuf() call. Nothing more. MSDN just says: "BUFSIZ is the
> required user-allocated buffer for the setvbuf routine." BUFSIZ is NOT
> a suggestion for a buffer size when dealing with large binary files.
> So the bug is in APR where BUFSIZ is used for something it was not designed for.
>
> This has already been discussed for years on this list, but nothing has changed:
> http://subversion.tigris.org/servlets/ReadMsg?listName=dev&msgNo=82087
> http://subversion.tigris.org/servlets/ReadMsg?listName=dev&msgNo=113549
>
> So it is clearly an apr and not a windows issue.

Providing a patch might help: They define a buffer size of their own
(APR_FILE_BUFSIZE, defined to 4096 on Win32) which should probably be
used for binary file copies since it's used for buffered files as
well.

Though 16k seemed to perform best on Win32 a year or 2 ago, when
tested by an svn dev; while no negative impact was detected on Unix
systems when shrinking from larger sizes to this relatively small
buffer size. All this was done on local filesystems.

HTH,

Erik.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Tue Dec 18 12:05:16 2007

This is an archived mail posted to the Subversion Dev mailing list.