Hi,
Below are some plans for improving WC performance that I'd like to get
done for 1.3 (or 1.4, depending on the schedule for 1.3 and how much time
I can put into it). Assuming no objections, I'll start working on this
ASAP. maxb: we need to keep track of each other for your #525 work.
The aim of this proposal is to make svn status faster by eliminating
much of the overhead caused by having to read all property files many
times, and by unnecessarily slow property change detection. While
here, I also want to do something about some other problems with property
handling:
- Allow GUI clients to show the svn:needs-lock state without having to
read each props file.
- Avoid using up lots of inodes with redundant files.
NOTE. Much of this is out-lined in notes/wc-improvements; I'm just
describing what I'm planning to do and fleshing out some details. I'm
not claiming any inventorship here:-)
I want to do the following things:
- Store a flag in the entries file indicating whether there are any
property modifications to this entry.
- Not store any props file if there are no property modifications
(use the prop-base instead).
- Cache the presence of svn:special and svn:needs-lock properties
in the entries file. (And possibly other svn: props if that makes any
speed difference.)
The props-mod flag:
Since the svn libraries are the only ones supposed to touch the .svn
area, we don't need to rely on the OS filesystem to keep track of
whether there are property modifications or not. This is different
from text mods, which are expected to happen outside of our control.
Keeping a flag in the entries file indicating whether there are
property mods or not avoids lots of apr_stat calls. These are known
to be slow on some systems, and they're not free anywhere.
No props file in the no mods case:
Currently, we store two versions of each property file, one working
and one base file. Since most files in a typical WC have no prop mods,
this is a lot of redundancy. Storing only the prop-base file in the
no mods case will avoid lots of typically small files. Since the
entries file know whether there are prop mods, the WC library will
know to read the props-base file in that case.
Caching certain properties:
"svn status" currently needs to know, for each file, if the
svn:special property is set. This requires reading each property file,
which seems like a lot of work. To avoid this, the presence of this
property can be cached in the entries file.
UI clients often want to show the svn:needs-lock property so the user
immediately knows that a file will need locking. Currently, this will
require reading all property files (or scanning the directory to check
the read-only state, but that's less accurate.) Since we're
introducing a mechanism to cache properties in the entries file, we
might as well cache svn:needs-lock to make it available via the
svn_client_status call.
Caching other svn: properties might also be useful. For example,
caching svn:ignore and svn:externals would avoid reading the property
files for each directory. I will need to do some experimentation to
see if this becomes a bottle-neck or not. Note that I only plan to
cache the presence of properties, not the values themselves. That's
to avoid bloating the entries file totally. NOte that for boolean
properties, the presence of the property is equivalent to a true value.
All this caching of properties and the props-mod flag (or whatever it
will be called) will make the entries files larger. I assume,
however, that reading a few extra disk blocks from the same file is
much faster than opening or stating lots of small files.
How to do this:
These changes require a WC format bump. We need to keep the old code
for old WCs, since we can't always upgrade to a new format. I'm not
even sure its worth the trouble to upgrade old WCs.
Since I'll be messing with the WC format, I want to do this work on a
branch. Also, this will make it possible to ship 1.3 should it be
ready without making this work block a release.
Regards,
//Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Thu Jul 28 23:31:55 2005