John Szakmeister <john@szakmeister.net> writes:
> How do I get 'next-key' value from the repo then? I mean, right now I'm
> building a cursor into the repo, and close it when I'm done iterating. Or,
> are you saying I should create a function to get called by retry_txn() that
> will take a current key and return the next-key? Perhaps a NULL value for
> the current key will mean to grab the first key and return it?
We store a row in the table whose key is "next-key" (or the symbol
svn_fs_next_key_key). So, you might need a new function down in
bdb/reps-table.c, perhaps svn_fs_bdb__rep_next_key(), to fetch the
value of this key.
>
> > 3. Call svn_fs__prev_key()* on that value (this will be "the current
> > key"). 4. Try to fetch the representation at the current key. If you
> > fail, no sweat.
> > 5. But if you succeed, read the rep data (which checks the checksums).
> > 6. If "the current key" != "0", goto step 3 and repeat, otherwise,
> > you're done.
> >
> > So, at most, the only time you spend in a given trail is the time
> > taken to fetch a single representation, or the time taken to read a
> > chunk of data from it.
> >
> > Questions? Comments? Are you ready to throw up your hands and quit
> > yet? (please say no -- you've been a terrific sport through this)
> >
> > :-)
>
> Nah, I like SVN... I'm here to stay. :-)
Well, that's certainly great to hear.
> Sounds easy enough. :-) Although, while we're talking about some of
> the pros and cons over one method versus another, I have a slight
> issue with something. For me, I'd like to know that when I type
> 'svnadmin verify' that everything checks out okay. If we only walks
> the reps table and check checksums, all we are doing is verifying
> that the data for a file is correct. Technically, there could be a
> problem in the node representation and you won't have a clue until
> you do an 'svnadmin dump' or try to access that particular node
> revision.
HahaHAHAha. I don't know *how many* times I've started to type this
same concern to you, but didn't.
See, the original issue report complains that a repository admin won't
know about brokenness in his repository until running svnadmin dump.
So, the reaction was to say, "Oh, we need 'svnadmin verify', and it
should verify checksums." But my respose to that (which I apologize
for not airing sooner) was that if you're going to write a subcommand
specifically about verifying the data in your repos, it really should
check everything, not just file contents. But here's the thing --
'svnadmin dump' already effectively does this, so why write another
subcommand?
Let's walk through the design of the perfect verifier. There are
various ways to do this -- various levels at which we can slip into
the filesystem, requiring various degrees of trust that at least
*something* in the filesystem is infallible. Here are a few
suggestions, with some pro/con commentary.
(a) for revision in revisions:
- crawl the tree under that revision, verifying dirs and files
for txn in txns:
if not txn.committed:
- do the same tree check
# This algorithm would cover verification of the tree structure of
# the filesystem, both for committed and unfinished transactions
# (though, we could drop the checks of unfinished transactions).
# The major problem with the algorithm is that because of our DAG
# filesystem layout, we'll be doing a **TON** of unnecessary
# checks. So, we revise.
(b) for revision in revisions:
- crawl the tree under that revision, verifying dirs and files
that were created or modified in this revision
for txn in txns:
if not txn.committed:
- do the same tree check
# Okay, now we aren't doing so much redundant verification. This
# is a definate improvement, but isn't really a full solution.
# Why? We haven't verified copies or change records yet, both of
# which are crucial to 'svnadmin dump'. Revise.
(c) for revision in revisions:
- read the changes associated with this revisions's underlying txn
- crawl the tree under that revision, verifying dirs and files
that were created or modified in this revision, and
checking that non-bubble-up changes and copies are represented
in the changes list
for txn in txns:
if not txn.committed:
- do the same tree check
# Getting better, getting better. This algorithm doesn't verify
# 'delete' change records (since you can't "see" a deletion by
# walking a tree). I'm not sure exactly how to verify deletions,
# save maybe checking that path@revision+1 either doesn't exist, or
# does exist as a new thing.
As you can see, to come even close to doing verification correctly,
you effectively must verify each tree under each revision, preferably
without verifying the same data more than once. But when it's all
said and done, this is pretty much what 'svnadmin dump' *does*.
I dunno, dude. I'm tempted to recommend that we not add the code
clutter of this extra verification step. My gut instinct is: if we're
writing whole new codepaths just to answer the "Would my dump fail?"
question, and those codepaths are gonna take every bit as long to run
as just running 'svnadmin dump', we're wasting our time and effort.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Tue Aug 5 17:28:31 2003