Julian.
I want to draw your attention to something that you may or may not realize
about designing for the BDB backend. The 'trail' objects are (generally)
representative of Berkeley DB transactions -- that part I'm sure you know.
But you might not realize the value of keeping transactions as small as
possible. Berkeley DB will accumulate locks (which I believe are
page-level, not as tight as row-level like you might hope) over the course
of a transaction, releasing those locks only at transaction commit/abort.
Berkeley DB backends are configured to have a maximum number of locks and
lockers allowed, and it's easier than you might think to hit the max-locks
thresholds (especially under high concurrency) and see an error (typically a
"Cannot allocate memory") result from that.
For example, in the loop below you are writing a bunch of rows to the
`changes' table. Could be 10. Could be 100,000. 100,000 writes and
associated locks might be a problem or it might not. But I use it as a way
to encourage you to think about reducing the amount of work you spend in any
one trail while doing your obliterate work.
(And yeah, this information should probably live somewhere. Maybe you could
drop it into trail.h as a comment?)
julianfoad_at_apache.org wrote:
> Author: julianfoad
> Date: Mon Dec 14 11:32:43 2009
> New Revision: 890279
>
> URL: http://svn.apache.org/viewvc?rev=890279&view=rev
> Log:
> Update the BDB "changes" table when starting an obliteration transaction.
>
> This is another step towards implementing "replace an old revision" to be
> used for basic obliteration in BDB. The "copies" and "node-origins" tables
> are still to do.
>
> * subversion/libsvn_fs_base/revs-txns.c
> (changes_dup): New function to make the new "changes" table entries.
> (txn_body_begin_obliteration_txn): Call changes_dup().
>
> Modified:
> subversion/trunk/subversion/libsvn_fs_base/revs-txns.c
>
> Modified: subversion/trunk/subversion/libsvn_fs_base/revs-txns.c
> URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_fs_base/revs-txns.c?rev=890279&r1=890278&r2=890279&view=diff
> ==============================================================================
> --- subversion/trunk/subversion/libsvn_fs_base/revs-txns.c (original)
> +++ subversion/trunk/subversion/libsvn_fs_base/revs-txns.c Mon Dec 14 11:32:43 2009
> @@ -434,6 +434,52 @@
> }
>
>
> +/* Duplicate all entries in the "changes" table that are keyed by OLD_TXN_ID,
> + * creating new entries that are keyed by NEW_TXN_ID.
> + *
> + * Each new "change" has the same content as the old one, except with the
> + * txn-id component of its noderev-id (which is assumed to have been
> + * OLD_TXN_ID) changed to NEW_TXN_ID.
> + *
> + * Work within TRAIL. */
> +static svn_error_t *
> +changes_dup(const char *new_txn_id,
> + const char *old_txn_id,
> + trail_t *trail,
> + apr_pool_t *scratch_pool)
> +{
> + apr_array_header_t *changes;
> + int i;
> +
> + SVN_ERR(svn_fs_bdb__changes_fetch_raw(&changes, trail->fs, old_txn_id, trail,
> + scratch_pool));
> + for (i = 0; i < changes->nelts; i++)
> + {
> + change_t *change = APR_ARRAY_IDX(changes, i, change_t *);
> +
> + if (change->kind != svn_fs_path_change_delete
> + && change->kind != svn_fs_path_change_reset)
> + {
> + const char *node_id, *copy_id;
> +
> + /* Modify the "change": change noderev_id's txn_id to NEW_TXN_ID */
> + node_id = svn_fs_base__id_node_id(change->noderev_id);
> + copy_id = svn_fs_base__id_copy_id(change->noderev_id);
> + SVN_ERR_ASSERT(svn_fs_base__key_compare(
> + svn_fs_base__id_txn_id(change->noderev_id), old_txn_id) == 0);
> + change->noderev_id = svn_fs_base__id_create(node_id, copy_id,
> + new_txn_id,
> + scratch_pool);
> + }
> +
> + /* Save the new "change" */
> + SVN_ERR(svn_fs_bdb__changes_add(trail->fs, new_txn_id, change, trail,
> + scratch_pool));
> + }
> + return SVN_NO_ERROR;
> +}
> +
> +
>
> /* Generic transaction operations. */
>
> @@ -783,7 +829,8 @@
> * pervade node-ids throughout history. But what actually uses them, and
> * does anything use them during txn construction? */
>
> - /* ### TODO: Dup the "changes" that are keyed by the txn_id. */
> + /* Dup the "changes" that are keyed by the txn_id. */
> + SVN_ERR(changes_dup(new_txn_id, old_txn_id, trail, trail->pool));
>
> /* ### TODO: Update the "node-origins" table.
> * Or can this be deferred till commit time? Probably not. */
>
>
--
C. Michael Pilato <cmpilato_at_collab.net>
CollabNet <> www.collab.net <> Distributed Development On Demand
Received on 2009-12-14 16:19:40 CET