[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Severe repository database corruption

From: Frank Freund <ffreund_at_memocomp.de>
Date: 2004-07-02 15:04:27 CEST

Thanks *all* for the help, it seems that the permissions did the trick.
I will give a final feedback next week.

But what I don't understand, and that's why I didn't digged in this
direction yesterday is that the repository worked from its begin on, for
about 6 months and 400 commits with around 700MB, with *exactly the same
settings without any problem*.
So how to explain this? What triggered the error?

Frank

Aaron Wohl wrote:

>-I see you doing a lot of stuff as root... be sure when your done to do
>chmod -R user:group for who it should be owned by for the webserver or
>synserver
>
>-With the same sort of problems your seeing, I was able do svnadmin dump
>then svnadmin create and svnadmin load. Ill bet your stuff was
>recoverable without going to backup tapes.
>
>-during your work it looks like stuff was left as /usr/local/apache2/svn
>During recovery its important that noone else have access for stuff like
>this, I do mv /usr/local/apache2/svn /usr/local/apache2/svn.sick .
>Then do the repair work and put it back. Double check your
>owner/group/permission as that can make it look broken aftewards if root
>owns stuff.
>
>----- Original message -----
>From: "Frank Freund" <ffreund@memocomp.de>
>To: users@subversion.tigris.org
>Date: Fri, 02 Jul 2004 00:34:19 +0200
>Subject: Severe repository database corruption
>
>Hi all,
>today morning, and after my last commit yesterday (and noone accessed
>the repository in any way in between) worked fine, I got the following
>error message while trying to commit (using TortoiseSVN):
> > "DB_RUNRECOVERY: Fatal error, run database recovery."
>Checking the repository via the browser, I got: "Could not open the
>requested SVN filesystem", so I started reading docs, mailing lists and
>searched the web.
>To anticipate, none of the things I tried worked :-( and it seems, that
>the database is the problem (but maybe not the reason)
>
>First I will give some information about the environment, then the
>things I tried and finally some of the sources I found concerning (other
>people having) this problem.
>
>Environment
>-------------------
>The repository was running now for about half a year with nearly 400
>commits and 700MB repository size.
>The System configuration is:
>SuSE 8.2 with
>* Subversion 0.35.1 (now upgraded to 1.0.5, but the problem remains
>exactly the same)
>* BDB 4.2.52.1
>* Apache 2.0.48
>All above build from source, and the clientside
>* Tortoise SVN 0.24.0 build 554
>
>Attempts
>-------------------
>First I tried
> > linux: # svnadmin recover /usr/local/apache2/svn
> > Please wait; recovering the repository may take some time...
> > Recovery completed.
> > The latest repos revision is 394.
>which looked fine, but didn't worked - same errors as above. So next was
> > linux: # svnadmin verify /usr/local/apache2/svn
>which also completed successfully, but didn't helped either. Next step
>were the databasetools:
> > linux: # ./db_recover -h /usr/local/apache2/svn/db
>This worked, but I just could access the repository over the browser for
>some tries, before the same error occurred again. (reproducable always)
>Accessing the repository using TorsoiseSVN, I just tried checkouts,
>failed directly (reproducable always)
>Tried again:
> > linux: # svnadmin verify /usr/local/apache2/svn
> > svn: Berkeley DB error while opening environment for filesystem
>/usr/local/apache2/svn/db:
> > DB_RUNRECOVERY: Fatal error, run database recovery
>and
> > linux: # ./db_recover -h /usr/local/apache2/svn/db
>which worked again for short (sometimes 10 seconds, never longer than
>about 2 minutes). Some tries later the verify failed:
> > linux: # svnadmin verify /usr/local/apache2/svn
> > Verified revision 0.
> > svn: Berkeley DB error while beginning Berkeley DB transaction for
>filesystem /usr/local/apache2/svn/db:
> > DB_RUNRECOVERY: Fatal error, run database recovery
> > svn: Berkeley DB error while closing 'nodes' database for filesystem
>/usr/local/apache2/svn/memocomp/db:
> > DB_RUNRECOVERY: Fatal error, run database recovery
>
>Meanwhile updated to SVN1.0.5 and imported the backup (Jesus saves:-)
>successfully, but the symptoms remain the same.
>Some posts mentioned problems with mixed up libdb's, but as far as I
>know a
> > linux: # grep libdb /proc/17455/maps
> > 4002a000-400e3000 r-xp 00000000 08:02 42429
>/usr/local/BerkeleyDB.4.2/lib/libdb-4.2.so
> > 400e3000-400e5000 rw-p 000b8000 08:02 42429
>/usr/local/BerkeleyDB.4.2/lib/libdb-4.2.so
>with 17455 beeing a httpd-PID, confirms me that this isn't the case.
>
>The http-error log shows:
> > [Thu Jul 01 08:11:02 2004] [error] [client 192.168.100.11] Could not
>fetch resource information. [500, #0]
> > [Thu Jul 01 08:11:02 2004] [error] [client 192.168.100.11] Could not
>determine the proper revision to access [500, #160029]
> > [Thu Jul 01 08:11:02 2004] [error] [client 192.168.100.11]
>(13)Permission denied: Berkeley DB error while checkpointing after
>Berkeley DB transaction for filesystem /usr/local/apache2/svn/db:
> > DB_RUNRECOVERY: Fatal error, run database recovery [500, #160029]
> > [Thu Jul 01 08:11:02 2004] [error] [client 192.168.100.11]
>(20014)Error string not specified yet: Berkeley DB error while closing
>'nodes' database for filesystem /usr/local/apache2/svn/db:
> > DB_RUNRECOVERY: Fatal error, run database recovery
> > [Thu Jul 01 08:17:02 2004] [error] [client 192.168.100.11]
>(20014)Error string not specified yet: Berkeley DB error while opening
>environment for filesystem /usr/local/apache2/svn/db:
> > DB_RUNRECOVERY: Fatal error, run database recovery, referer:
>http://192.168.100.2/repository.html
>and later
> > [Thu Jul 01 20:14:46 2004] [error] [client 192.168.100.11] Could not
>fetch resource information. [500, #0], referer:
>http://192.168.100.2/repository.html
> > [Thu Jul 01 20:14:46 2004] [error] [client 192.168.100.11] Could not
>open the requested SVN filesystem [500, #160029], referer:
>http://192.168.100.2/repository.html
> > [Thu Jul 01 20:14:46 2004] [error] [client 192.168.100.11] (17)File
>exists: Could not open the requested SVN filesystem [500, #160029],
>referer: http://192.168.100.2/repository.html
> > [Thu Jul 01 20:14:47 2004] [error] [client 192.168.100.11]
>(20014)Error string not specified yet: Berkeley DB error while opening
>environment for filesystem /usr/local/apache2/svn/db:
> > DB_RUNRECOVERY: Fatal error, run database recovery, referer:
>http://192.168.100.2/repository.html
>
>Further readings
>-------------------
>Repository recovery:
>http://svnbook.red-bean.com/svnbook/ch05s03.html#svn-ch-5-sect-3.4
>Another broken BDB:
>http://www.contactor.se/~dast/svnusers/archive-2004-05/0556.shtml
>Corrupted repository db:
>http://www.contactor.se/~dast/svn/archive-2004-05/0071.shtml
>Corrupted repository db:
>http://www.contactor.se/~dast/svn/archive-2004-05/0097.shtml
>Unrecoverable databases:
>http://www.contactor.se/~dast/svnusers/archive-2004-05/1283.shtml
>http://subversion.tigris.org/servlets/BrowseList?list=users&by=thread&from=202660
>
>So that's all and I like to thank everyone who has read this &%&$*long
>post.
>
>Thanks in advance for any help!
>Frank
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
>For additional commands, e-mail: users-help@subversion.tigris.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Fri Jul 2 15:07:02 2004

This is an archived mail posted to the Subversion Users mailing list.