Re: corrupted FSFS repository

Hi all,

On Wed, 2006-08-02 at 05:31 -0500, John Szakmeister wrote:
> On Sunday 05 February 2006 10:55, Marc Sherman wrote:
> > John Szakmeister wrote:
> > > If you have the original repository handy, this is actually very easy
> > > to fix. You can send me a private email if you need that done, but it
> > > looks like you've already gotten back up and running using a
> > > different solution.
> >
> > John, do you or Malcome have a tool (or instructions) for repairing
> > these corrupted repositories that's publicly available? I've been
> > living in fear of this bug hitting the repositories I manage at work for
> > months now. :)
>
> Have no fear. It's at least possible to recover with no data loss (at least
> that's what I've experienced with the ones that I have personally fixed...
> after I found the magic recipe).
>
> I've attached my script. Your probing motivated me to automate the fix of
> this problem. :-)

In less than a month, I've now been hit twice with a corrupted
repository. First the techy bits:

        Platform: Linux, Ubuntu Breezy (2.6.12-10-686-smp).
        Subversion: 1.3.0
        Access via: https / mod_dav_svn
        Subversion store: FSFS
        Disk mirroring on the repository, no errors reported.

The corruption has come from the same user, Windows XP, TortoiseSVN
1.3.1.

I cannot, unfortunately, give access to the repos or the rev causing the
grief. The best I can do is try to summarize what one of these commits
looks like:

      * very large commits (latest 83 MB, previous 400 MB)
      * with a mix of large & small files
      * upwards of 120 files & folders
      * long pathnames (up to 169 en-US characters)
      * mix of characters (including spaces, parens, tildes)
      * a variety of file formats: PDF, CAD diagrams, zip files, Word
        docs, Excel spreadsheets
      * slowish network connection
      * other folk with faster network connection committing at same
        time.

I've tried the fsfsverify.py script (provided kindly by John Szakmeister
to this list back on Feb 8, 2006) with the following results (both
times, same error).

Results of svnadmin verify repos
---------------------------------
...
* Verified revision 3183.
svnadmin: Unexpected end of svndiff input

Results of svnadmin dump --incremental -r 3184 repos > /tmp/x
----------------------------------------------------------------
svnadmin: Unexpected end of svndiff input

        I do have a post-commit hook that does an incremental dump and
        the results of that are identical to the results of the dump
        done above. So, I guess that means that the corruption does
        occur during the commit process and not something after that
        commit has occurred.

Results of fsfsverify.py --fix-read-length-line-error repos/db/revs/3184
------------------------------------------------------------------------

Traceback (most recent call last):
  File "../fsfsverify.py", line 683, in ?
    process(noderev, rev_file, options.dump_instructions,
options.dump_windows)
  File "../fsfsverify.py", line 636, in verify
    dump_windows)
  File "../fsfsverify.py", line 273, in verify
    digest = parse_svndiff(f, self.length, dump_instructions,
dump_windows)
  File "../fsfsverify.py", line 162, in parse_svndiff
    (instr_size, data_length, target_length_tmp) = \
  File "../fsfsverify.py", line 82, in parse_svndiff_instruction
    raise "Invalid instruction found at offset %d" % start
Invalid instruction found at offset 81484719

I'd absolutely love to help in getting this solved. If there's anything
else that I can provide, please let me know.

-- 
rick@onnadayr.ca
IT Firefighting and Prevention
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org