Re: apr_file_copy performance on Win32

"Peter N. Lundblad" <peter@famlundblad.se> wrote in message
news:Pine.LNX.4.55.0411022054280.8648@fnapp.local...

> I don't think BUFSIZ is the modern way of choosing a buffer size. Usually
> one uses st_blksize from a stat call, but that seems not to be available
> in APR.

I don't think anthing 'stat' is particularly standard.

I would say

#define APR_COPY_BLOCK_SIZE 65536

would be as good a way of dealing with this as any.

> We have a special version of file copying already. Using our own loop
> wouldn't be much code duplication. So, if this is a problem, we could do
> this as a short-term solution.

I don't think I follow this paragraph!

Are you saying that there's an equivalent copy function in SVN which is
sometimes used instead of the APR version?

There does actually need to be a little bit of thought into the block size -
in my very limited benchmarking here, copying a 8Meg-ish file, you get the
following results:

512byte blocks - about 2 seconds
4096byte blocks (the default using fwrite/fread) - about 1 second
102400byte blocks (that's SVN_STREAM_CHUNK_SIZE) - about 800-900ms.
any power of two sized block from about 16384 upwards - about 350ms.

CopyFile does a copy in 65536byte chunks and takes around 350ms.

Although tiny blocks are slow for obvious reasons, there is clearly some
other pathology here for larger block sizes, probably an interaction with
underlying file-system block sizes (clusters, etc)

However, I'd be suprised if 65536 turned to be a bad choice on any system,
and I can't believe it would be worse than 512!

> Ofcourse, getting it into APR is even
> better. Have you asked the APR people?

I haven't dared. Would a change like this make it into APR and through to
SVN in any human time-scale?

Will

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Tue Nov 2 21:37:45 2004