[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: FSFS optimization

From: Blair Zajac <blair_at_orcaware.com>
Date: 2007-09-19 16:09:09 CEST

On Sep 19, 2007, at 6:18 AM, Malcolm Rowe wrote:

> On Mon, Aug 13, 2007 at 03:18:59PM -0700, Dan Christian wrote:
>> The first thought is to work on caching some of these: "node.0.0" is
>> the node revision, and "props" contains the revision properties. The
>> "changes" file is always(?) appended to, so we might be able to keep
>> it open and keep appending to it. The tricky bit is that the httpd
>> will close at some point and an a fresh one will end up advancing the
>> transaction. This implies that the disk version must be kept up to
>> date.
> This is the key, actually. NFS only guarantees close-to-open cache
> coherency, so we need to ensure a file written by one process/host is
> closed before it's opened by another one... and there unfortunately
> isn't any explicit FS call that says "I'm finished with this
> transaction
> on this process for the minute, feel free to flush state to disk now".
> And so we end up flushing to disk after _every_ operation, which
> sucks.
>> An alternative/additional thought would be to move the transactions
>> directory to local disk so that the OS can cache more aggressively.
>> This would require network load balancers to always direct the same
>> client to the same server.
> I think this is the way to go. I had a patch to do this, I think,
> though I don't think I had thought through everything, so there
> might be
> lurking issues. (Also, from a UI point-of-view, how do you switch it
> on?)

I need long lived transactions that are visible to multiple systems,
so definitely don't want to make this the only place where
transactions are built. I would like to see a configuration for fsfs
that would specify where the transaction directory is.

>> A completely separate idea is to work on caching revision files on
>> local disk (so the full text version doesn't have to be regenerated
>> repeatedly).
> +1. A server-side rep cache sounds like an excellent idea.
> (Again, I wonder about the configuration, though. Maybe FSFS needs
> some
> equivalent to the DB_CONFIG file... or maybe svnadmin should have an
> fsfstune command?)

A DB_CONFIG like file sounds easier to code, just to edit a text
file. But maybe you want to be able to edit that file in a live
repository, so you would need an svnadmin fsfstune that would allow
you to change values and move transactions in progress from one
directory to another.


Blair Zajac, Ph.D.
Subversion training, consulting and support
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Wed Sep 19 16:09:34 2007

This is an archived mail posted to the Subversion Dev mailing list.