[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Towards a native-FS-backed filesystem

From: Mark Benedetto King <mbk_at_lowlatency.com>
Date: 2004-03-30 20:35:25 CEST

On Tue, Mar 30, 2004 at 12:44:52PM -0500, Greg Hudson wrote:
> On Tue, 2004-03-30 at 07:49, Garrett Rooney wrote:
> > As for how to structure this work so that it doesn't disrupt the main
> > thrust of Subversion development, I'm thinking that a good first step
> > would be to modify libsvn_fs such that it makes use of a vtable for all
> > calls into the underlying filesystem implementation,
>
> Yeah, that will have to be done at some point, of course, though it's
> not necessary for prototyping. Since CMike has volunteered to do this
> work, I am happy to let him take point on it.
>
> It does appear that there are two different levels where an FS
> implementation might wish to diverge from the current one--at the
> libsvn_fs/bdb level of making associations, or at the more abstract
> libsvn_fs level. Interestingly, kernel filesystems have the same
> problem; at least in *BSD, a lot of ufs code is shared between several
> filesystems (ffs, mfs, lfs, ext2fs) but there is also a higher-level vfs
> abstraction used by more radically different filesystems like nfs.

It believe that this code sharing should be accomplished via libraries
of common subroutines rather than multiple strict API tiers. This
would take away the design-time burden, and allow more iterative
implementation and refinement.

Way down underneath covers of, say, libsvn_fs_sql, there may well be
a need for another abstraction layer to hide the intricasies of the
various SQL dialects and access mechanisms, but that's a different
story.

> I don't care much whether we do the work on a branch or on the trunk
> with a disabling #ifdef. One requires more merging work and one
> potentially puts vestigial code into releases (or requires us to remove
> code in dist.sh, which some might find distasteful), but neither cost is
> very high.

I'd prefer the trunk, with a configure-time flag to enable it;
lots of other projects release experimental code, we shouldn't be
any different.

> CMike wrote:
> > IMO, a libsvn_fs filesystem backend is a toy project (sorry if others
> > disagree), but the necessary design modifications overlap with a
> > feature (pluggable DB backends) that is widely desired.
>
> I've observed a significant audience of people who want a
> low-administration, easily understood back end, and who don't trust
> databases much. We shall see. :) Fortunately, we don't need agreement
> on this point.

I am among this group (even though I rely heavily on databases for
all sorts of things, I have full-time database administrators keeping
them running).

> Ben Collins-Sussman wrote:
> > My initial gut reaction: I find this overview very clever. I'm really
> > curious about the proposed single-file format to hold an entire
> > revision. There's a bit of hand-waving going on here: this file
> > would have to contain a whole tree structure plus multiple node-revs,
> > and each node-rev would have to contain text and prop-deltas. I
> > wonder: would this magical file format look something like a
> > version-3 dumpfile? :-)
>
> There could be similarities, but there would also have to be big
> differences:
>
> * The FS has to use skip-deltas, whereas a version-3 dumpfile uses
> only single-rev deltas.
>
> * The FS needs to be better indexed; in particular, it needs a much
> more developed conception of directories than "here are some files which
> were added to this directory in rev N."
>
> * Because of the way transactions are built, all the file node-rev
> data needs to preceed the directory data in a revision file.
>

It might also be worthwhile to use a format that can be parsed/written
more efficiently.

--ben

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Tue Mar 30 20:35:45 2004

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.