[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Berkeley DB corruption - 'changes' table (WORSE)

From: Paul L. Suh <psuh_at_apple.com>
Date: 2003-10-03 05:38:14 CEST

Folks,

OK, the repository is seriously toast. After restoring from a backup,
we tried to check in additional changes and ran into tons of problems.
Individual files keep causing repository deaths with errors like:

subversion/libsvn_fs/bdb/strings-table.c:101: (apr_err=160010)
svn: Filesystem has no such string
svn: locate_key: no such string `h6w'

Worse, it's inconsistent throughout the repository. Most files work.
Some files don't, consistently. Occasionally, if we delete a file from
the head of the tree, add back a small file (like only a few bytes),
then replace it in the wc with the real (~1.5-2 MB file) and do a
check-in it works. Other times it doesn't. The error code is
consistent, but the 3 letters `h6w' are not -- other times they are
`f8r' or other letter-number-letter combos. The repository dies the
same way when I run 'svnadmin dump' on it, even after running 'svnadmin
restore'.

Our previous backup to that was a few days back thanks to a messed up
tape rotation. :-P At least that one seems clean.

BTW is there an Issue Tracker item on this? Or should I open one?

--Paul

On Oct 2, 2003, at 2:35 PM, Paul L. Suh wrote:

> Karl,
>
> Yes, the admin did turn off httpd before doing the recover.
>
>
> --Paul
>
>
> On Oct 2, 2003, at 1:36 PM, kfogel@collab.net wrote:
>
>> When the person ran 'svnadmin recover', did they observe the cautions
>> mentioned in
>>
>> http://subversion.tigris.org/project_faq.html#wedged-repos
>>
>> ... especially the part about shutting off Apache and any other
>> process that might conceivably access the repository?
>>
>> -Karl
>>
>> "Paul L. Suh" <psuh@apple.com> writes:
>>> Folks,
>>>
>>> Subversion 0.24.2 (Yes I know it's old, we're planning on updating to
>>> 0.30 in 2 days, just as soon as the current project phase
>>> finishes. Curse you, Murphy!) We're getting the error:
>>>
>>> "Berkeley DB error while opening `copies' table for filesystem
>>> /Library/Repository/MacOSXTroubleshooting/db: Invalid argument "
>>>
>>> Three questions:
>>>
>>> 1) Is there any way to recover? (Not a screaming necessity since we
>>> have daily backups, but there were a goodly number of commits
>>> yesterday since we're getting close to a release.)
>>>
>>> 2) Have there been any changes in the BDB table structure in more
>>> recent versions that would make this problem unlikely to happen?
>>>
>>> 3) Is there an Issue Tracker entry for this? I looked but couldn't
>>> find one.
>>>
>>> Additional detail follows.
>>>
>>>
>>> --Paul
>>>
>>> Paul L. Suh
>>> Trainer/Curriculum Developer
>>> Apple Computer
>>> 1892 Preston White Drive
>>> Reston, VA 20191
>>>
>>> (703) 264-3216
>>> http://train.apple.com/
>>>
>>>
>>> Original error e-mail from user:
>>>
>>>> Hi,
>>>>
>>>> Any idea how to fix this?
>>>>
>>>> A little while ago, svn failed due to a full disk on
>>>> my machine:
>>>>
>>>> [da0703a-dhcp110:Current/Production_Materials/Guide]
>>>> ellend% svn update 09_FileInternetShareing.fm
>>>> Restored 09_FileInternetShareing.fm
>>>> subversion/libsvn_wc/log.c:294: (apr_err=155009)
>>>> svn: Problem running log
>>>> svn: in directory ''
>>>> subversion/libsvn_subr/io.c:216: (apr_err=28)
>>>> svn: No space left on device
>>>> svn: svn_io_copy_file: error copying
>>>> '.svn/tmp/text-base/09_FileInternetShareing.fm.svn-base'
>>>> to '09_FileInternetShareing.fm.tmp'
>>>>
>>>> I emptied my trash to free up space.
>>>>
>>>> Now, when I try to check in a file I get this:
>>>>
>>>> [da0703a-dhcp110:HDEssentials/Current/Production_Materials]
>>>> ellend% svn ci Guide/09_FileInternetShareing.fm
>>>> subversion/libsvn_wc/lock.c:116: (apr_err=155004)
>>>> svn: Attempted to lock an already-locked dir
>>>> svn: working copy locked:
>>>> /Users/ellend/Desktop/HDEssentials/Current/Production_Materials/
>>>> Guide
>>>> svn: run 'svn cleanup' to remove locks (type 'svn help
>>>> cleanup' for details)
>>>>
>>>> So then I try to run svn cleanup, and I get this:
>>>>
>>>> [da0703a-dhcp110:HDEssentials/Current/Production_Materials]
>>>> ellend% svn cleanup
>>>> subversion/libsvn_wc/log.c:294: (apr_err=155009)
>>>> svn: Problem running log
>>>> svn: in directory 'Guide'
>>>> subversion/libsvn_wc/log.c:1145: (apr_err=155009)
>>>> svn: start_handler: error processing command
>>>> 'modify-entry' in 'Guide'
>>>> subversion/libsvn_wc/log.c:487: (apr_err=155009)
>>>> svn: error getting file affected time on
>>>> `Guide/.svn/props/09_FileInternetShareing.fm'
>>>
>>>
>>> svnadmin recover produces:
>>>
>>> Acquiring exclusive lock on repository db.
>>> Recovery is running, please stand by...
>>> Recovery completed.
>>> subversion/libsvn_fs/bdb/bdb-err.c:61: (apr_err=160029)
>>> svn: Berkeley DB error
>>> svn: Berkeley DB error while opening `copies' table for filesystem
>>> /Volumes/data/Repository/MacOSXTroubleshooting/db:
>>> Invalid argument
>>>
>>> Doing a direct db_recover results in:
>>>
>>> sudo /opt/BerkeleyDB.4.1/bin/db_recover -v
>>> db_recover: Finding last valid log LSN: file: 4311 offset 581049
>>> db_recover: Recovery starting from [4311][578974]
>>> db_recover: Recovery complete at Thu Oct 2 08:30:57 2003
>>> db_recover: Maximum transaction ID 8000000b Recovery checkpoint
>>> [4311][581049]
>>> db_recover: Recovery complete at Thu Oct 2 08:30:57 2003
>>> db_recover: Maximum transaction id 80000000 Recovery checkpoint
>>> [4311][581049]
>>>
>>> But svnadmin recover after that still produces the same problem.
>>>
>>> Systems involved are Mac OS X 10.2.6 on client and server, I built
>>> the
>>> entire 0.24.2 tarball from source, Berkeley DB 4.1.25, httpd 2.0.46
>>> custom built from source, all access via http.
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
>>> For additional commands, e-mail: users-help@subversion.tigris.org
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
>> For additional commands, e-mail: users-help@subversion.tigris.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Fri Oct 3 05:39:48 2003

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.