Pristine text storage - status

Pristine text is now stored in the "Pristine Store" accessed by
svn_wc__db_pristine_...(), if you build with
"CFLAGS=-DSVN_EXPERIMENTAL_PRISTINE".

With one more significant change (in the attached patch) it is "close"
to working.

Things keeping it in "experimental" mode:

  * It's still suffering a few test failures - see below.

  * The WC upgrade code isn't written - see below.

  * The removal of unreferenced texts from the store is incomplete.

Things that are not blocking:

  * Switch from using MD-5 checksums as the key, to using SHA-1.

    (I think that's a nice separate task that we can do afterwards, or
even in the middle of everything else. I have made the code store both
checksums when installing a new pristine text and accept either kind of
checksum when looking up an existing pristine text.)

PROBLEMS AND TEST FAILURES
==========================

Here's what happens running the test suite (with the attached patch):

[[[
FAIL: db-test: Unknown test failure; see tests.log.
  - The test tries to add a BASE_NODE checksum without first
    installing the pristine text in the store, which now fails
    an assertion.

FAIL: basic_tests.py 40: automatic conflict resolution
  - File A/D/G/rho ends up empty after 'resolve' and 'update'.

FAIL: diff_tests.py 4: replace a file with a file
  - Diff output is blank for file A/D/G/rho; should show a mod.

FAIL: externals_tests.py 1: test checkouts with externals
  - workqueue.c:1918: assertion failed (src_stream != NULL)
  - a file external?

FAIL: externals_tests.py 15: should not be able to mv or rm a file external
FAIL: externals_tests.py 16: place a file external into a directory external
FAIL: externals_tests.py 18: binary file externals
FAIL: externals_tests.py 23: tag directory with file externals from wc to url

FAIL: prop_tests.py 9: props work when reverting a replacement
  - reverting a replacement
  - due to entry_modify()?

FAIL: revert_tests.py 3: revert a replaced file with no properties
FAIL: revert_tests.py 5: revert svn cp PATH PATH replace file with props
FAIL: revert_tests.py 6: revert a merge replacement of file with history
FAIL: revert_tests.py 7: revert svn cp URL PATH replace file with props
FAIL: revert_tests.py 8: revert file after second replace
FAIL: revert_tests.py 18: reverted replace with history restores checksum

FAIL: special_tests.py 7: merge symlink into file
]]]

Experience with squashing the earlier failures suggests they will all be
caused by a rather smaller number of issues. Some of the main causes
are:

  * Some problem related to file-externals doesn't work.

  * Calls to entry_modify, on a node that is replaced, lose the checksum
recorded on the base node, and thus lose track of its pristine text.

UPGRADING OLD WORKING COPIES
============================

The next step (which can be done in parallel with fixing the above
problems) is upgrading old WCs. I have a plan for the procedure to
upgrade an old WC:

  1. For each pristine text that is present in the old location: if its
checksum is not stored in the DB row for that node, calculate and store
its MD-5 checksum in the DB row for that node. (Or SHA-1 if we've
switched to using SHA-1 already).

  2. For each pristine text that is present in the old location: install
a copy of it in the Pristine Store (which includes calculating the other
checksum) and then delete it from the old location.

  3. Bump the WC format number to indicate the upgrade is functionally
complete.

If anyone wants to tackle any of these, that would be great. I'll be
around this week and on holiday next week.

- Julian