Re: UpPgrading BDB repositories on Win32

Branko ÄŒibej <brane@xbc.nu> writes:

> I'm fiddling with the README.txt file for the 1.2 Win32 binaries,
> because I want to include instructions for upgrading BDB repositories
> (1.1.x uses BDB 4.2, 1.2.x uses BDB 4.3). I've come up with the
> following procedure:
>
> ----------
> In this release, we've upgraded BerkeleyDB from version 4.2.52 to
> 4.3.27. If you are currently using Subversion 1.1.x as a server on
> Windows with BerkeleyDB, use the following steps to upgrade your
> repositories:
>
> -- Make sure nobody is using the repository (stop Apache/svnserve,
> restrict access via file://).
> -- Using the old (1.1.x) binaries, run "svnadmin recover" for
> each repository.
> -- Using the new (1.2.x) binares, for each repository:
> -- run "svnlook youngest"
> -- run "svnadmin recover"

The BDB docs say that 4.3 undates the log file format, and that seems
to imply that the 1.2.x recover is not required.

>
> Once again, this is only necessary for repositories based on BDB.
> You do NOT have to dump/reload your repositories.
> ----------
>
> This seems to work in my tests. However, I'm seriously puzzled by the
> need to run "svnlook" before "svnadmin recover" with the new
> binaries. Recover by itself will choke, with the following error:
>
> ----------
> Repository lock acquired.
> Please wait; recovering the repository may take some time...
> svnadmin: DB_RUNRECOVERY: Fatal error, run database recovery
> svnadmin: bdb: Program version 4.3 doesn't match environment version
> svnadmin: bdb: Skipping log file repo/db\log.0000000001: historic log version 8
> svnadmin: bdb: repo/db\log.0000000002: log file open failed: No such file or directory
> svnadmin: bdb: PANIC: No such file or directory
> svnadmin: bdb: DB_ENV->log_put: 2: DB_RUNRECOVERY: Fatal error, run database recovery
> ----------
>
> I'm inclined to call this a bug in svnadmin recover, but I'm not sure
> how to start fixing it.

This problem is not specific to Windows, it also occurs on my Linux
box. While investigating I discovered that BDB (4.3 at least) will
sometimes call bdb_error_gatherer() but not go on to return an error
status. When this happens an svn_error_t is leaked. I'm using the
following patch which causes those errors to be displayed, (but
perhaps we should just clear them?):

Index: subversion/libsvn_fs_base/bdb/bdb-err.c
===================================================================
--- subversion/libsvn_fs_base/bdb/bdb-err.c (revision 14573)
+++ subversion/libsvn_fs_base/bdb/bdb-err.c (working copy)
@@ -86,9 +86,9 @@
 svn_error_t *
 svn_fs_bdb__wrap_db (svn_fs_t *fs, const char *operation, int db_err)
 {
- base_fs_data_t *bfd;
+ base_fs_data_t *bfd = fs->fsap_data;
 
- if (! db_err)
+ if (! db_err && ! bfd->errcall_baton->pending_errors)
     return SVN_NO_ERROR;
 
   bfd = fs->fsap_data;
Index: subversion/libsvn_fs_base/bdb/bdb-err.h
===================================================================
--- subversion/libsvn_fs_base/bdb/bdb-err.h (revision 14573)
+++ subversion/libsvn_fs_base/bdb/bdb-err.h (working copy)
@@ -76,7 +76,7 @@
 #define SVN_BDB_ERR(ec_baton, expr) \
   do { \
     int db_err__temp = (expr); \
- if (db_err__temp) \
+ if (db_err__temp || ec_baton->pending_errors) \
       return svn_fs_bdb__dberr ((ec_baton), db_err__temp); \
   } while (0)
 

Starting with a 4.2 repo:

$ svnadmin recover repo
Repository lock acquired.
Please wait; recovering the repository may take some time...

Recovery completed.
The latest repos revision is 31.

Using 4.3 I get a failure:

$ subversion/svnlook/svnlook youngest repo
../svn/subversion/libsvn_fs_base/bdb/bdb-err.c:76: (apr_err=160029)
svnlook: Berkeley DB error for filesystem repo/db while opening environment:
DB_VERSION_MISMATCH: Database environment version mismatch
../svn/subversion/libsvn_fs_base/fs.c:316: (apr_err=0)
svnlook: bdb: Program version 4.3 doesn't match environment version

Running 4.3 recover I get another failure:

$ subversion/svnadmin/svnadmin recover repo
Repository lock acquired.
Please wait; recovering the repository may take some time...
../svn/subversion/libsvn_fs_base/bdb/bdb-err.c:59: (apr_err=160029)
svnadmin: Successful return: 0
../svn/subversion/libsvn_fs_base/fs.c:316: (apr_err=0)
svnadmin: bdb: Program version 4.3 doesn't match environment version
../svn/subversion/libsvn_fs_base/fs.c:316: (apr_err=0)
svnadmin: bdb: Skipping log file repo/db/log.0000000002: historic log version 8

4.3 still fails:

$ subversion/svnlook/svnlook youngest repo
../svn/subversion/libsvn_fs_base/bdb/bdb-err.c:76: (apr_err=160029)
svnlook: Berkeley DB error for filesystem repo/db while opening environment:
Successful return: 0
../svn/subversion/libsvn_fs_base/fs.c:316: (apr_err=0)
svnlook: bdb: Skipping log file repo/db/log.0000000002: historic log version 8

Trying again, 4.3 now works:

$ subversion/svnlook/svnlook youngest repo
31

However running 4.3 recover still fails:

$ subversion/svnadmin/svnadmin recover repo
Repository lock acquired.
Please wait; recovering the repository may take some time...
../svn/subversion/libsvn_fs_base/bdb/bdb-err.c:59: (apr_err=160029)
svnadmin: Successful return: 0
../svn/subversion/libsvn_fs_base/fs.c:316: (apr_err=0)
svnadmin: bdb: Skipping log file repo/db/log.0000000002: historic log version 8
../svn/subversion/libsvn_fs_base/fs.c:316: (apr_err=0)
svnadmin: bdb: Skipping log file repo/db/log.0000000002: historic log version 8
../svn/subversion/libsvn_fs_base/fs.c:316: (apr_err=0)
svnadmin: bdb: Skipping log file repo/db/log.0000000001: historic log version 8
../svn/subversion/libsvn_fs_base/fs.c:316: (apr_err=0)
svnadmin: bdb: Skipping log file repo/db/log.0000000002: historic log version 8

and that causes svnlook to fail again:

$ subversion/svnlook/svnlook youngest repo
../svn/subversion/libsvn_fs_base/bdb/bdb-err.c:76: (apr_err=160029)
svnlook: Berkeley DB error for filesystem repo/db while opening environment:
Successful return: 0
../svn/subversion/libsvn_fs_base/fs.c:316: (apr_err=0)
svnlook: bdb: Skipping log file repo/db/log.0000000002: historic log version 8

Once again, if I repeat the operation it works:

$ subversion/svnlook/svnlook youngest repo
31

The upgrade procedure that seems to work on my test repository is:

- run 'svnadmin recover' using BDB 4.2
- remove db/_db.00?
- remove or archive db/log.*

Then just start using BDB 4.3, there is no need to run a 4.3 recover
although it doesn't do any harm. Note that I have to remove all the
log files, even those not designated "unused", I don't know under
what circumstances this is safe.

-- 
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org