Re: gen-make.py

On Thu, Oct 24, 2002 at 11:54:29AM -0500, Ben Collins-Sussman wrote:
> Karl Fogel <kfogel@newton.ch.collab.net> writes:
> > Ben Collins-Sussman <sussman@collab.net> writes:
>...
> > Even in this respect, if you consider the usual alternative: many
> > systems require developers to remember to update a statically
> > maintained list whenever they add a new C file.
>
> Which is the norm. It's the difference between *one* developer having
> to remember to augment a static list, vs. *all* developers having to
> remember to re-run their generators. I don't like the burden being
> spread out like that... it doesn't make sense to me. People already
> run "./configure; make" all the time. I'm tired of teaching newbies
> about the non-standard "gen-make.py build.conf; ./configure; make"
> sequence.

Yes, that is non-standard. But you can also tell them the standard way:

$ ./autogen.sh
$ ./configure
$ make

Using gen-make.py directly is an optimization to skip some autogen steps.
That is the approach that I've always advocated, but you can always return
to recommending autogen.sh.

Note that autogen.sh is also needed in automake-based projects when files
are added, so our need for it when files arrive isn't new/extraordinary.

Also note that gen-make and autogen are only for people working direct from
the repository. People using tarballs don't need to know about either.

> > Our system generates that list automatically, [...]
>
> ...which feels a tad sloppy. If I copy a temporary .c file into the
> working copy, I'd hate to see it accidentally picked up by the build
> scanner.

You've never raised this issue, and it hasn't ever come up in the 16 months
or so that we've been using gen-make. I'll round-file this point :-)

> Step back for a second. Here have this nice system with a beautiful
> .conf file that allows you to be specific about *so* many behaviors,
> and yet this system *doesn't* have inherent knowledge about what files
> to compile for any given thing. Doesn't that strike you as odd?

If you would like, you can list the specific files in build.conf. It totally
supports that style. IMO, that would be a mistake.

The concept of "compile all .c files in this directory" for this link-unit
has worked quite well for us.

> > Both ways are error prone, but at least our way you don't have to go
> > crawling over the logs to figure out how to update the list when
> > something goes wrong.
>
> Actually, I don't think this is a realistic fear. I don't see how a
> developer could add a new C file and forget to edit the static list;
> otherwise the developer would never be able to compile at all. And we
> all know that while perhaps we don't always run 'make check' before
> each commit, we all *do* compile our changes first. :-)

It is absolutely a realistic fear. How many people here have added files and
not updated the .dsp files? Every single one of us. Multiple times. Oh, here
and there, some developer with a particularly large brain on that day will
remember to edit the .dsp file. But I'd say the 90% is to forget it.

The gen-make system is designed to keep both build systems in sync.
Automatically.

Back to the original query, it would be possible to insert an extra step to
deal with added files. Specifically, we could have gen-make build a
"structure" and then the ./configure script would flesh that out by
inserting whatever set of .c files it finds in specific dirs at configure
time.

Consider the following example from build-outputs.mk today:

libsvn_test_DEPS = subversion/tests/svn_test_editor.lo subversion/tests/svn_tests_main.lo subversion/libsvn_delta/libsvn_delta-1.la subversion/libsvn_subr/libsvn_subr-1.la
libsvn_test_OBJECTS = svn_test_editor.lo svn_tests_main.lo
subversion/tests/libsvn_test-1.la: $(libsvn_test_DEPS)
        cd subversion/tests && $(LINK) -o libsvn_test-1.la $(libsvn_test_OBJECTS) ../../subversion/libsvn_delta/libsvn_delta-1.la ../../subversion/libsvn_subr/libsvn_subr-1.la $(SVN_APRUTIL_LIBS) $(SVN_APR_LIBS) $(LIBS)

It could be possible to copmute the _DEPS and _OBJECTS at configure time if
we told some processor "for libsvn_test, look in subversion/tests/*.c".
Thus, the set of source files is automatically updated at configure time.

But, again, I would not advocate that choice. This means that you must run
the full-on ./configure script to get file additions/removals reflected in
the build system. Today, a developer can:

$ svn add path/to/new/file.c
$ ./gen-make.py build.conf
$ make local-all

If we altered it, then you'd have to:

$ svn add path/to/new/file.c
$ ./configure
$ make local-all

And I can tell you which is going to be faster or slower :-)

Note that "classic" systems also run some of the overhead of ./configure. In
those systems, a develop adds a .c file to a Makefile.in and then runs
config.status to regenerate Makefiles. We could conceivably hook into the
config.status processing for finding the .c files, thus avoiding the bulk of
configure.

By splitting the system into two steps, that also means you *need* those two
steps. Changes to build.conf must be run during autogen.sh *and* during
config processing.

In the current model, we have two independent pieces: build-outputs.mk which
consists of config-independent build info (build by autogen.sh) and
Makefile(.in) which consistes of config-dependent build info, and it selects
the appropriate parts from build-outputs. We can make lots of changes to
build.conf and never need to run any part of the config process.

And lastly, think back to the people who are affected by this: people
working from source control. Developers. You can tell them to run the
autogen sequence after each pull, or the configure (or config.status) after
each pull, but they'll forget to do it sometimes. Or think they won't need
to (I always avoid re-running configure/config.status in httpd or apr(util)
and get bit, just like you're talking about.

Changing the mechanics of the system will not change the social aspect which
is the root of the problem (run X after updating).

And, of course, changes in the system might lose or complicate the .dsp
generation capability. (not there yet, but getting closer every day...)

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org