On Mon, Nov 24, 2003 at 08:53:13PM -0000, kfogel@tigris.org wrote:
> Translation: somewhat to our surprise, there isn't a simple fix for this,
> because the things that we thought were major inefficiencies turn out not to be
> all that costly.
>
> So, moving this out of 0.34 and thus out of 1.0. Our performance on huge files
> may not be as good as it could be, but it's already better than CVS. We don't
> need to block the 1.0 release on further improvement.
>
One thing to consider: it's possible that the current strategy of
keeping fulltexts of HEAD contributes to the problem; we need to
do a whole bunch of inserting in order to put the fulltext of a
big file in, and (asynchronously now, but still relevant) a whole
bunch of deleting in order to deltify the previous revision against
the new fulltext. This sort of churn is the kind of thing that
databases are generally bad at.
It is possible the storing *forward* skip-deltas in the repository
would be a big win in this regard (it also makes some conceptual
sense: "record these changes to the repository"). When a commit
comes in, just write the delta and you're done. This means
the i/o associated with a commit is proportional to the size of
the change, which is certainly a nice feature (it would REALLY help
svn import, right?).
Yes, this would cause updates/checkouts of HEAD to need to do
significant computational work in order to construct the fulltext,
but I suspect that this inversion to "append-only" might actually
improve the overall BDB performance (less fragmentation of the repo
leading to smaller overall repos leading to better native-filesystem-cache
performance, and an append-only database opens up the possibility of very
aggressive access concurrency).
I should note that this is yet another one of Greg Hudson's ideas that
I wish were my own. However, in his defense, the half-bakedness is all
mine.
--ben
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Mon Nov 24 23:33:45 2003