[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

[PATCH] Reduce the amount of WriteFile() syscalls when writing to buffered files

From: Evgeny Kotkov <evgeny.kotkov_at_visualsvn.com>
Date: Mon, 21 Aug 2017 18:45:24 +0300

Hi everyone,

Currently, apr_file_write() can cause an excessive amount of syscalls
for buffered files on Windows in some situations. This happens because
for buffered files, writing is implemented with a loop that keeps copying
the data to the internal 4KB buffer and writing the contents of this
buffer to disk with WriteFile(). In other words, performing a 100 KB
write currently results in 25 consecutive WriteFile(4096) syscalls.

This patch series reduces the amount of syscalls in such situations by
performing a single WriteFile() call without any buffering, if possible.
If some data is already buffered, then the buffer is first filled, flushed
and the remaining part of the data is written with a single WriteFile().

My measurements indicate that writing in larger chunks and avoiding
such syscalls can save a certain part of the CPU time. For example,
"svn import" does both small and large buffered writes with the following
pattern:

    write: nbytes = 9
    write: nbytes = 5
    write: nbytes = 86267
    write: nbytes = 9
    write: nbytes = 5
    write: nbytes = 86413
    write: nbytes = 9
    write: nbytes = 5
    write: nbytes = 86415
    write: nbytes = 9
    write: nbytes = 5
    write: nbytes = 67680
    ...

When I test "svn import" for a large file before and after this patch, I
see that

  - the amount of syscalls decreases from 199,403 to 17,061 and

  - the overall CPU time decreases from 7.28 s to 6.68 s. (I think that's
    a lot, considering that "svn import" does a many other CPU-intensive
    operations.)

The implementation is split into three dependent patches. The first two
patches lay the necessary groundwork by factoring out a couple of helper
functions. The third patch is the core change that implements the
described optimization.

The log messages are included in the beginning of each patch file.

Thanks,
Evgeny Kotkov

Received on 2017-08-21 17:45:57 CEST

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.