On Fri, Apr 04, 2003 at 09:15:14AM -0800, Tom Lord wrote:
>
>
> [This is based on some assumptions about the implementation that
> I haven't verified so I apologize in advance if it's completely
> bogus.
It isn't bogus at all. :-)
>...
> Currently, you use BDB's native logging format, right? What I
Yes.
>...
> It seems to me that there are two kinds of log-based recovery you're
> relying on:
>
> (1) BDB's built-in recovery facilities for interrupted
> transactions --- just enabling ACID properties
Yup. This is satisfied if you copy the data files *first*, *then* copy the
log files. The backed-up copy will retain its integrity.
(it may need to be db_recover'd, but I'm not sure; regardless, it is
guaranteed to be recoverable to *some* state)
Of course, it is entirely possible that you might lose a commit that was not
fully flushed. But you'll have a solid repository up to revision N-1.
> (2) user-directed recovery from bugs in svn (including bugs
> in BDB) --- starting from a believed-good backup and
> replaying txns as far forward as possible without
> reintroducing corruption
This is why you want *all* the log files. Even those which are no longer
"open". You can replay the logs forward/backwards to different checkpoints.
> For (1), using native BDB logs is the only practical way to go -- but
> you really don't need much logged data for that: as soon as there are
> no pending transactions and you sync to disk, any existing logs should
> be candidates for removal at that point.
Correct. The "db_archive" command shows you which log file are no longer
participating in pending transactions. Those log files can them be removed
if you don't anticipate the "replay" thing. Or they can be archived to
offline storage in case you *do* need to replay the state of the db to
various points in time.
In effect, our unversioned revision properties are versioned by the BDB log
files :-)
> For (2), presumably the idea here is that when "something goes wrong",
> you grab a back-up of a believed-good old database, and then play as
> much of the logs going forward from that as you can without
> reintroducing the problem.
Correct.
> If (2) uses BDB logs, then yes, log file cycling and the back-up
> schedule have to be linked together -- otherwise you'll have gaps in
> the log record and won't be able to recover past those gaps.
Correct.
> This
> requirement, because of the nature of BDB logs, leads to a situation
> where the log files can be several times larger than the database
> itself -- yet discarding them without backing them up introduces new
> risks.
Yup. The risk here is that something was changed that you don't want to
happen (such as a change to an unversioned property), or that you want to
pretend that the application did not perform some action (a very difficult
form of 'svn obliterate').
You can't really use this technique to work around application bugs,
however, since replaying the [BDB] logs will simply replay the buggy
database transactions.
If you had application-level logs (as you suggets below), then you
definitely have more flexibility in the types of replay that are possible.
> Some observations: Log cycling sufficient for (1) could be fully
> automated and space consumption be reasonably bounded. Users should
> not need to interact with log files for purpose (1) at all. They
> should not even need to back them up.
The problem is that SVN cannot take it upon itself to define this policy.
We've already heard from users who are preserving logs on purpose (i.e. they
are fully aware of the use of those logs).
Discarding data, by default, does not seem like a winning strategy.
That said, I *do* think we can adjust the post-commit template to provide an
example for cleaning out the logs after each commit. It would be commented
out, of course, but an administrator will quickly learn about the issue. The
admin can also encode their particular policy into a new set of on-disk
templates for any repository construction.
> BDB logs are inappropriate and inefficient for (2). Forward-only
Now we move into the opinion portion of the discussion :-).
> logging (rather than (a), symmetric logging) is sufficient and cuts
> space in half.
BDB logs are probably forward-only, but don't quote me on that. I suspect
that you would need to restore to state N (db files + open logs) and then
start replaying logs to reach a given state. If you *can* replay backwards,
then "neat" :-)
> Higher-level logging (at the level of svn txns rather
> than BDB) (rather than (b), page (or key/value) logging) would make
> the contents of the log less dependent on storage management layers,
> thus less likely to themselves contain bad data in the event that
> "something goes wrong".
Agreed. It also gives you the possibility to fix the problem at the
application level before performing the replay.
> Furthermore, transaction data (essentially the stream of requests from
> the client) should be small -- about as small as it can be made. (I
Yup. It could essentially be a compressed tree patch.
> know that long ago, clients sent full-text and more recently they send
> deltas -- I'm not sure how "complete" that change is at this time --
Actually, the clients have "always" sent deltas. It's been a long while, but
the server->client delivery of deltas is newer. (i.e. we were exactly the
opposite of CVS!)
> but in the long run, the stream of write txns from the client is
> essentially just fancy changesets, right?)
Correct. And those are your app-level logs.
> So what I would suggest is that you separate concerns (1) and (2)
> above. Use tightly-space-bound, auto-managed BDB logs to enable ACID
> behavior from BDB -- but users can be oblivious to that.
Uh oh. Here it comes :-)
Yes, this would be a neat facility. I'm not sure how to mesh this behavior
in with BDB's logging facility. e.g. can we get BDB to retain disk-level
integrity, but avoid logs, and store the app-level logs ourselves? Of
course, then an administrator can also display the app-level logging to
conserve disk space.
I suspect that we cannot disable BDBs logging. Otherwise, we wouldn't have
the "new feature" of doing just that coming down the pipe. Until that point,
I'm not sure that duplicating [the net result of] BDB's logging with an
app-level log will be helpful.
>...
> Such a journal will be database-technology-independent -- the same
> journal mechanism applies to an SQL database or any other style.
> Roughly speaking, journaling will double the storage size of
> repositories -- but that's all it will do. Users will have to tie
> backup scheduling for the database and the journal together -- but
> they won't have a circumstance where if backups are infrequent the
> journal grows to several multiples the size of the database.
Agreed.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sat Apr 5 00:02:08 2003