Hi Greg. Thanks for reviewing.
On Tue, 2011-02-15, Greg Stein wrote:
> On Tue, Feb 15, 2011 at 10:06, Julian Foad <julian.foad_at_wandisco.com> wrote:
> >...
> > * ==== Operating Procedures ====
> > *
> > * The steps should be carried out in the order specified. (See rationale.)
> > *
> > * * To add a pristine, do the following inside an SDB txn:
> > * * Add the table row, and set the refcount as desired. If a row
> > * already exists, add the desired refcount to its refcount, and
> > * preferably verify the old row matches the new metadata.
> > * * Create the file. Creation should be fs-atomic, e.g. by moving a
> > * new file into place, so as never to orphan a partial file. If a
> > * file already exists, preferably leave it rather than replace it,
> > * and optionally verify it matches the new metadata (e.g. length).
> > *
> > * * To remove a pristine, do the following inside an SDB txn:
> > * * First, check refcount == 0, and abort if not.
> > * * Delete the table row.
> > * * Delete the file or move it away. (If not present, log a
> > * consistency error but, in a release build, return success.)
> > *
> > * * To query a pristine's existence or SDB metadata, the reader must:
> > * * Ensure no pristine-remove txn is in progress while querying it.
>
> What does "pristine-remove txn" mean?
It means the SDB txn defined under "To remove a pristine" above. The
obvious way to ensure it's not in progress would be to do this query in
an SDB txn with a suitable lock (and I think the default lock type that
you get in a txn would be sufficient).
> > *
> > * * To read a pristine text, the reader must:
> > * * Ensure no pristine-remove txn is in progress while querying and
> > * opening it.
> > * * Ensure the pristine text remains in the store continuously from
> > * opening it for the duration of the read. (Perhaps by ensuring
> > * refcount remains >= 1 and/or by cooperating with the clean-up
> > * code.)
>
> Taking out a WC lock would prevent a pristine from being deleted, right?
Yes, that would work, because of the rule below that requires an
exclusive WC lock in order to purge an unreferenced text.
(Originally I didn't say that here because I'm trying to keep the spec
as two separate layers, first presenting this low-level spec for
maintaining consistency between the files and the PRISTINE table rows in
the store, and secondly under the heading "Reference Counting" a higher
level spec for maintaining the references to pristines, which layers on
top of this one. The latter is where WC locks are involved. However, a
cross-reference here would be helpful.)
Is a WC lock on the root of the WC an exclusive lock, in that it could
not co-exist with any other WC lock whether on the root or deeper? I'm
unsure of the semantics of WC locks.
> >...
> > * ==== Notes ====
> > *
> > * * This procedure can leave orphaned pristine files (files without a
> > * corresponding SDB row) if Subvsersion crashes. The Pristine Store
> > * will still operate correctly. It should be easy to teach "svn cleanup"
> > * to safely delete these. ### Do we need to define the clean-up
> > * procedure here?
> > *
> > * * This specification is conceptually simple, but requires completing disk
> > * operations within SDB transactions, which may make it too inefficient
> > * in practice. An alternative specification could use the Work Queue to
> > * enable more efficient processing of multiple transactions.
>
> Note that my initial design for the pristine inserted a row which
> effectively said "we know about this pristine, but it hasn't been
> written yet". The file would be put into place, then the row would get
> tweaked to say "it is now there". That avoids the disk I/O within a
> sqlite txn.
That would indeed avoid disk I/O within any txn during "add". The
"remove" operation would go through the same states in reverse, I
assume. But this seems to imply an extra complexity. How would the
whole thing work - locking for "add", "remove" and read operations;
crash recovery? Maybe it's straightforward but I'll have to write it
out in detail if you think it's worth it.
I would suggest first we need to assess whether a file move within a txn
is in fact a problem.
> >...
> > * === Invariants in a Valid WC DB State ===
> > *
> > * * No pristine text, even if refcount == 0, will be deleted from the store
> > * as long as any process holds any WC lock in this WC.
> > *
> > * The following conditions are always true outside of a SQL txn:
> > *
> > * * The 'checksum' column in each NODES table row is either NULL or
> > * references a primary key in the 'pristine' table.
> > *
> > * * The 'refcount' column in each PRISTINE table row is equal to the
> > * number of NODES table rows whose 'checksum' column references this
> > * pristine row.
>
> ACTUAL_NODE also holds references. Shouldn't these count towards the checksum?
One day, yes, but for 1.7 no. We don't use that column yet.
> > *
> > * The following conditions are always true
> > * outside of a SQL txn,
> > * when the Work Queue is empty:
> > * (### ?) when no WC locks are held by any process:
> > *
> > * * The 'refcount' column in a PRISTINE table row equals the number of
> > * NODES table rows whose 'checksum' column references that pristine row.
> > * It may be zero.
>
> hmm? duplicate text?
Could be... I was losing concentration at this end of the doc.
> > *
> > * ==== Operating Procedures ====
> > *
> > * The steps should be carried out in the order specified.
> > *
> > * * To add a pristine text reference to the WC, obtain the text and its
> > * checksum, and then do this while holding a WC lock:
> > * * Add the pristine text to the Pristine Store, setting the desired
> > * refcount >= 1.
> > * * Add the reference(s) in the NODES table.
> > *
> > * * To remove a pristine text reference from the WC, do this while holding
> > * a WC lock:
> > * * Remove the reference(s) in the NODES table.
> > * * Decrement the pristine text's 'refcount' column.
> > *
> > * * To purge an unreferenced pristine text, do this with an *exclusive*
> > * WC lock:
> > * * Check refcount == 0; skip if not.
> > * * Remove it from the pristine store.
>
> This seems a bit duplicative of the "Operating Procedures" in the first section.
This is supposed to be the higher layer of spec, defining how to add a
new pristine text to the WC ALONG WITH ADDING A REFERENCE TO IT in the
NODES table, in terms of the lower layer "add this text to the store"
operation.
- Julian
Received on 2011-02-16 13:43:00 CET