[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: major design changes...can we

From: Mark C. Chu-Carroll <mcc_at_watson.ibm.com>
Date: 2002-05-18 19:28:12 CEST

>
> Interesting.
>
> A fairly large part of your use case for the developer already exists -
> It's called Eclipse&CVS. I'm using these 2 tools currently for my Java
> development, and they do most of what you want (diff/merge/ignore .bak
> files, etc.)
>
> As far as the build management goes, that's more of a front-end tool
> question than a repository issue. What it does need is for the
> repository to support atomic commits and changesets.
> BitKeeper and Arch are built around these concepts. Not sure to what
> extent SVN supports them (haven't been here long.)

Actually, there's a whole lot to gain by integrating build support into
the SCM system.

The main factor is that by building things into the system, you
can do automated work that would be intractable for a human build
manager.

For example... Andreas Zeller did some work with changeset based
systems on identifying problems. The basic idea is that a bunch
of programmers all checked in changes. Then, during the nightly
build, you discover that the system no longer compiles correctly,
or that it no longer passes the standard tests.

But there's 30 changesets. That means 30 tests to determine if
one of those changesets is the one that broke the build. But what
if what breaks the build is a *combination* of the changes in more than
one changeset?

Zeller's system did a binary-search like process to try to determine the
minimal group of changesets that cause the breakage.

A system like that could be implemented outside of the system; but it's a
heck of lot nicer to tie enough of it into the system that it can be done
fully automatically.

One thing that one of the Stellation team members wants to look at is a
way of handling this problem for open-source. Rather than working with
patch queues, you set up your project like this:

- One branch, called "main". This is where the canonical current version
  of the system lives.

- For each programmer, there are two branches: user-work, and
user-commit.

- A programmer works mainly in user-work. They frequently merge changes
  from main into user-work. When they finish a set of changes that they'd
  like to check in, they do a checkin to user-commit.

- Every night (or every N hours), a change tester runs:
  - It first checks to make sure that each new version in user-commit
    compiles and tests. Any that don't automatically send a note to the
    programmer describing the breakage.
  - Then, it tries to merge the full set of new versions in user-commit
    branches. If it fails, it does a binary search to determine the
    minimal set of things that break the compile. Those are now flagged
    as broken, and the programmers are notified that their changes cause
    breaks when combined with the others in the minimal break set.
  - Then, it runs the tests of the merge of the remaining unbroken
    user-commit. If the tests fail, it does the binary search to
    find the minimal breakage set, and notifies the offending programmers
    of what tests failed.
  - Then it takes the set of things that work, and checks them into
    main.

Another major issue that you can address with integration of build with
repository is product cacheing. If you're working with a really big
system, builds take a *long* time. And lots of programmers are doing
builds to test their own stuff. All of the programmers are spending a lot
of time idle waiting for build results. (At one point, I was involved
with the VisualAge C++ project. Builds of the system could take two or
three hours. We'd spend half of our work day waiting for builds to
finish!)

But almost all of that build time is redoing the exactly same work on
many different systems. We had about 7 people in our building, working on
a part of a 4 million line system. Each person would be changing one or
two source files, and then waiting for the build result of those changes
compared with the nightly builds.

If the SCM system understands the build process, it can store the
intermediate results, and before starting a build step, check if anyone
else has either done that already, or is in the process of doing it. At
the least, you avoid a lot of redundant builds; at the best, you get free
build parallelization.

        -Mark

> Then with a decent tool the builder could pick and choose which
> changesets he/she needed and keep updating his/her repository to
> achieve consist builds. Each successful build could be
> "tagged/branched". Developers could compare the branches with their
> local builds to see what changesets would need to be applied/unapplied
> to match the build "branch".
>
>
> ----- Original Message -----
> From: "Dean Hiller" <dhiller@avaya.com>
> To: <dev@subversion.tigris.org>
> Sent: Friday, May 17, 2002 10:36 PM
> Subject: major design changes...can we
>
>
>> Hello all,
>> I have been developing my own source control system and just foud
>> out
> about subversion today. My system still seems quite orthogonal to
> yours believe it or not. I would like to see one of two things
> happen.....(if possible)
>> 1. see these features eventually in subversion
>> 2. somehow combine projects(this is no small task seeing how
>> orthogonal
> the two SCM's are)
>>
>> The main feature I would like to see is a viewable build map(attached
>> is
> the concept I am centering the system around). This could be side by
> side with a file version map. Each node on the file version map maps
> to 1 or many nodes on the build map. To see how much more orthogonal
> my system is(and some of the other radical ideas), go to the following
> website
>>
>> http://sourceforge.net/docman/display_doc.php?docid=9859&group_id=46703
>>
>> Is there any interest? Of course, I did not see anywhere on the
>> website
> what language your code is in? Is it in Java?
>>
>> Please reply directly to me also as I don't know if I successfully
> subscribed to the list.
>>
>> thanks for listening,
>> Dean
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------------->> ----
>
>
>
>
>
>
> ---------------------------------------------------------------------------->> ----
>
>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
>> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org

-- 
*** Mark Craig Chu-Carroll,  <mcc@watson.ibm.com>
*** IBM T.J. Watson Research Center
*** The Stellation project:
http://domino.research.ibm.com/synedra/synedra.nsf
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sat May 18 19:29:13 2002

This is an archived mail posted to the Subversion Dev mailing list.