Testing server/client back-compatibility: status report

Subversion 1.5 introduces a number of features with
backwards-compatibility concerns:

[a] --depth

    1.5 servers emulate pre-1.5 -N using the new depth machinery; 1.5
    clients emulate --depth against old servers by doing client-side
    filtering.

[b] merge tracking

    Merge tracking is both server and client-side; I'm not very
    familiar with the details of what should happen if you use merge
    tracking commands against old servers.

[c] brand new server-side features like setting revprops during commit

    These are the types of new features we're most used to; they don't
    cause much compatibility concerns because either they work or they
    error out quickly.

Before we release Subversion 1.5, we should be able to test that our
backwards-compatibility actually works. We should be able to run old
test suites with old clients against new servers, and new test suites
with new clients against old servers. As an example of what can go
wrong if we don't, Karl tracked down a bug revealed by running the
1.4.x test suite against a trunk svnserve which corrupted the working
copy. (And then fixed it :-) )

The Good News: Once we backport r25607, the 1.4.x test suite will pass
against a trunk svnserve! (I haven't tried the DAV clients and
servers yet.)

The Bad News: Running the trunk test suite against a 1.4.x server is
not so easy. As of r25648, you can tell the test suite that you're
running an old server:

   $ cd trunk
   $ svnserve1.4.x -d -r `pwd`/subversion/tests/cmdline
   $ make svncheck SERVER_MINOR_VERSION=4

and it will make some accommodations (create unsharded repositories,
skip revprop-commit tests, etc). It's pretty easy to deal with the
type [c] concerns this way. Unfortunately, I haven't dealt with
everything, and the test suite does not yet pass cleanly against a
1.4.x server: it fails 22 tests in 6 files. It's a combination of:

* Actual problems in our backwards-compatibility code (I'm pretty sure
  that the failures I'm seeing in depth_tests.py are shortcomings in
  the client)

* I have no idea what the proper behavior of merge tracking commands
  should be against an old server

* Other random things (some bugfixes involving 'svn status -u' which
  perhaps could be backported, etc)

Also, we could probably use some tests that *only* run against old
servers: for example, it would be nice to explicitly test that
running 'svn commit --revprop' throws an error against an old server
instead of silently ignoring the revprop argument and committing
without them.

Unfortunately, I'm about to vanish into the ThesisCave for a few weeks
and can't deal with this myself. I urge other folks to take up this
task --- there's a lot of low-hanging fruit here, and it's pretty
important that we are comfortable about backwards-compatibility before
we can release 1.5.

Here's the tests.log output from my latest execution of trunk
tests and client against 1.4.x branch svnserve, if you'd like to just look
for the failures without running it yourself:

http://people.csail.mit.edu/glasser/trunk-client-1.4-server-tests.log.txt

--dave

-- 
David Glasser | glasser_at_mit.edu | http://www.davidglasser.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org