[] small damages of the DB data could have fatal consequences but may go
unnoticed a long time...
-> always use "svnadmin verify" before doing backups!
I "forced" a very small damage in the server repository DB data,
and tried to find out what consequences this will have on
the whole svn content.
Small damages can go completely unnoticed for a long time,
but will prevent doing a complete and sucessfull "svnadmin dump"
and then may also prevent access to large ranges of revisions or
whole folders.
Only regular use of 'svnadmin verify' or 'svnadmin dump'
will show you these errors early !
I strongly recommend that the use of 'svnadmin verify'
is announced and recommended more clearly in the documentation,
and that its functionality is extended to 'double check' every bit of data
in the db for a possible corruption.
I also think that -if- a corruption is detected there should be a lot more
options
to recover and repair the corrupted data. currently svn will simply abort
and may therefore even prevent you from accessing undamaged data
beyond the real damage. Verify should be able to print a detailed list
of what is damaged, and a "repair" function should be able to
recover-dump the remaining data and maybe just mark/tag its property as
"damaged/recovered"
I also think that the dump format and database format itself is not yet
enough protected against (transmission) errors.
The data itself is protected with MD5 numbers,
but the actions and properties (uuid, date, filenames) seem to be
completely unprotected.
Since a dump/load cycle usually transfers MANY gigabytes of data
the chance that a bit gets corrupted is not that low.
There might also be "quickhack" tools to "filter" things from the
dump files and introduce such unnoticed property-damages.
In comparison to CVS, SVN ofcourse is much safer, because at least
it will -detect- a data error instead of returning defective data.
but if data gets damaged in CVS there are probably only one or two files
affected, and other file/folder content remains still readable.
====
c.a.t.
========
An example experiment as a Win32 .bat "[receipe]"
( please feel free to replace the "searchreplace" command with your
favourite tool
to perform a binary file search&replace )
@echo off
REM # cleanup
rmdir /s/q Repos
rmdir /s/q Work
REM # new repository and co WC and cd into that WC
svnadmin create Repos
mkdir Work
cd Work
svn co file:///D:/user/TestSVN/SVNCorrupter/Repos
cd Repos
echo "this is a very important number: 0123-4567-89AB-CDEF" >important.txt
svn add important.txt
svn ci important.txt -m "added important.txt"
REM # comitted revision 1
REM # change/commit the content of that file a few more times
echo "this is a very important number: 9876-5432-10AB-CDEF" >important.txt
svn ci important.txt -m "modified important.txt"
REM # comitted revision 2
echo "this is a very important number: 6666-7777-10AB-CDEF" >important.txt
svn ci important.txt -m "modified important.txt again"
REM # comitted revision 3
svnadmin list-unused-dblogs D:/user/TestSVN/SVNCorrupter/Repos
REM # no logs should be produced here, otherways delete them (e.g |xargs
rm )
cd ..
cd ..
REM # now 'BE' a bad thing, e.g. a defect hard disk block,
REM # defect ATA cable, defect backup, defect ram ...
REM # and do the following:
REM # search&replace ALL occourences of "0123-4567" within
REM # the DB folder by "DEAD-BEEF".
searchreplace D:/user/TestSVN/SVNCorrupter/Repos/db "0123-4567" "DEAD-BEEF"
@echo off
REM # go into Work folder
cd Work
REM # [important]
REM # delete the Repos,
REM # otherways svn is smart and uses files from .svn to get the last
revision
rmdir /s/q Repos
svn co file:///D:/user/TestSVN/SVNCorrupter/Repos
REM # notice what happens? nothing. evrything is fine? -REALLY?-
REM # so what about that previus revision?
REM # probably a few month or years with or without backups
REM # may pass before anyone tries to retrieve an old revision:
svn co -r 1 file:///D:/user/TestSVN/SVNCorrupter/Repos
REM # svn: Checksum mismatch on rep '1':
REM # expected: 4aa2e495635b6ea20f3a478832811999
REM # actual: 1801d855189000f14961a4d7227433bd
REM # now you would recognize that you are in big trouble...
REM # probably you would try a recover?
svnadmin recover D:/user/TestSVN/SVNCorrupter/Repos
REM # (several hours may pass on large repos...)
REM # Recovery completed.
REM # The latest repos revision is 3.
svn co -r 1 file:///D:/user/TestSVN/SVNCorrupter/Repos
REM # svn: Checksum mismatch on rep '1':
REM # expected: 4aa2e495635b6ea20f3a478832811999
REM # actual: 1801d855189000f14961a4d7227433bd
REM # ofcourse we are STILL in trouble.
REM # recover -does NOT- (and can not?) detect the checksum errors.
REM # because that's what "svnadmin verify" does!
svnadmin verify D:/user/TestSVN/SVNCorrupter/Repos
REM # (several hours may pass on large repos...)
REM # svn: DB_RUNRECOVERY: Fatal error, run database recovery
REM # svn: Checksum mismatch on rep '1':
REM # expected: 4aa2e495635b6ea20f3a478832811999
REM # actual: 1801d855189000f14961a4d7227433bd
REM # * Verified revision 0.
REM # svn: Checksum mismatch on rep '1':
REM # expected: 4aa2e495635b6ea20f3a478832811999
REM # actual: 1801d855189000f14961a4d7227433bd
REM # so, if we had run a verify before doing all our backups we
REM # could be fairly sure to have a working backup...
REM # but lets assume we did not know about svnadmin verify
REM # (its not mentioned very 'loud' in the documentation!)
REM # running db_recover on the Repos didn't change anything either...
REM # since the MD5 checksum is a feature of svn itself.
REM # so what now? ok, trying to dump/reload that whole thing
svnadmin dump >dump1
REM # * Dumped revision 0.
REM # svn: Checksum mismatch on rep '1':
REM # expected: 4aa2e495635b6ea20f3a478832811999
REM # actual: 1801d855189000f14961a4d7227433bd
REM # fine. this got us a fairly empty dump since it aborted on rev 1.
REM # so the only thing posible seems to do partial
REM # dumps up to the damage, skip the "destroyed"
REM # revisions and do a second partial dump.
REM # but the destroyed area might affect quite a
REM # few revisions.
svnadmin dump -r 2:3 D:/user/TestSVN/SVNCorrupter/Repos >dump2to3
REM # depending on where the damage occoured you may
REM # loose a single file revision, revision range,
REM # a whole file and history, or a whole subtree
REM # (think of a lost file move or lost file add).
cd ..
=======
c.a.t.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Sun Feb 29 03:31:17 2004