On Mon, 2002-05-20 at 11:24, Paul Smith wrote:
> %% "Mark C. Chu-Carroll" <mcc@watson.ibm.com> writes:
>
> >> So I can see how integrating build support into Stellation could be a
> >> win. You're already commited to understanding the semantics of the
> >> source tree. But with Subversion (and CVS) you lose a lot.
>
> mcc> Not necessarily. If you look at a system like ClearCase, they've
> mcc> kept artifact semantics out of the system, but they provided an
> mcc> integrated build mechanism which works extremely well. What they
> mcc> did was build a version of make which understands the
> mcc> repository. When clearmake is ready to issue a command, it does a
> mcc> versioned extension of that command, so that every input to the
> mcc> command is marked with a version label. If there's a file in the
> mcc> repository that was generated with the same version labels on all
> mcc> of its inputs, then it will use that as the result of the
> mcc> command; otherwise, it runs the command and caches its results.
>
> Just as an FYI, the reason the ClearCase can do this so nicely is very
> simple, and is not duplicated by any other SCM tool that I know of,
> including Subversion: their "workspaces" are actual filesystems.
>
> That is, ClearCase installs a new kind of filesystem into your kernel,
> called MVFS, which allows you to interact with the repository through
> normal filesystem commands. You "mount" your repository, then from
> within that mounted repository you say "cat foo@@/main/1" and you're
> using standard UNIX cat, but when the kernel dishes off the pathname
> "foo@@/main/1" to the filesystem (MVFS), it understands that it should
> break up this path and treat it as "element foo, version /main/1". This
> is very cool since you can use any program to access any version of the
> repository without needing to check out a temp copy, or any special
> "tool-aware" variant of the program.
>
> Anyway, leveraging this is how they get clearmake to work so well: what
> happens is clearmake tells MVFS "I'm about to run a command, so start
> remembering what happens". Then clearmake runs the command normally,
> using normal shell/compiler/whatever invocations. MVFS is, of course,
> receiving all those open(), read(), write(), etc. requests (that happen
> within the repository), and it is dutifully remembering them.
>
> Then when clearmake is done it asks MVFS to return what it remembered,
> and clearmake now knows exactly what files were opened during the
> invocation of the rule, what _versions_ of those files were used, etc.
> ClearCase stores this data in the repository and this provides
> hyper-accurate knowledge of exactly when targets are out of date,
> including both prerequisites _and_ versions; knowledge that make,
> relying solely on a stateless time-last-modified "database", cannot come
> close to duplicating.
I know about MVFS. It's a darned nifty trick. And I think it's a heck of
a lot harder to get audited builds to work if you can't intercept the
file system calls. But it is possible: non-filesystem based tools have
been able to accomplish similar things. For example, Vesta manages to
do audited builds quite elegantly, without having to pretend to be
a filesystem. Of course, their system is very heavyweight, and and
they've got things tied so closely together that they're largely
inseperable. But a Vesta-like approach to audited builds could work
in systems that aren't as monolithic as Vesta appears.
For example, take a look at some of the SCM interfaces that people have
written for the Ant build tool. You can get some really good audited
build support by building a derived product cache into your repository,
and tieing ant or make into it. Without the filesystem, you do have to
be careful, because you don't have as much information as ClearCase gets
from MVFS. But you can get darned close.
-Mark
PS: I suspect that it's probably time to move this discussion off-list.
If you'd like to talk more about it, I'll be glad to, but I'm afraid
I might wear out my welcome in subversion land if I take up any more
common bandwidth :-).
--
Mark Craig Chu-Carroll, IBM T.J. Watson Research Center
<mcc@watson.ibm.com>
*** The (recently renamed) Stellation project:
*** http://domino.research.ibm.com/synedra/synedra.nsf
*** GPG Public key available at keyserver.net
Received on Mon May 20 17:41:59 2002