On Sat, Dec 03, 2005 at 08:33:53PM -0500, John Szakmeister wrote:
> Very interesting. I think I'm going to have to digest that for a little while
> with some code in front of me though. I think I'll first try to recreate the
> corruption now that I realize I may have the data to do it.
Here's a concrete example, again using your r94.
We create the proto-rev file and write windows until we've got the
following (numbers at the start are file offsets):
0: DELTA 89 0 1051980\n
[APR file buffers flushed to disk here because we called apr_file_seek()
to get the offset of the svndiff data].
19: SVN\0
23: (first window starts here: 23 bytes past DELTA)
22304: (second window starts here: 22304 bytes past DELTA)
1248023: (last recognisable window [window number 96] starts here)
So after writing 305 4k blocks of svndiff data, we're at offset
1249299. We also have 1691 bytes of unwritten svndiff data buffered
(which is the remainder of window 96, by the way, as you can see if
you look at the duplicate set of windows later in the rev file).
Now we error while calculating window 97.
We start again, tacking the delta rep onto the end of the file, which is
still at offset 1249299, since we've not written out the rest of window
96 yet.
1249299: DELTA 89 0 1051980\n
1249318: SVN\0
1249322: (first window starts here: 23 bytes past DELTA)
1271603: (second window starts here: 22304 bytes past DELTA)
2497322: (window 96 starts here)
2500289: (window 97 starts here, so window 96 was 2967 bytes long)
2502904: (window 98)
2552326: (window 99, the last window)
2596801: ENDREP\n
2596808: noderev, etc.
Now the pool cleanup for the first open apr_file_t object occurs.
It writes the remainder of window 96 - the 1691 bytes of unwritten
buffered data - at that file's current file position: 1249299, overwriting
bytes up to and including offset 1250989, destroying the second DELTA
line, svndiff header, and window 1 (windows 2-99 are intact, however).
So we're left with two identical blocks, at offset 1691-1250989, and at
1250990-2500288, which is what we see.
What could we have been doing when we hit an error? We couldn't have
been writing the proto-rev file, because APR fills its 4k buffer
completely before writing it out (and note that APR versions prior to
0.9.7 didn't report errors when writing buffered files at all).
Since we wrote out all of window 96 (though didn't flush it all to disk),
I suspect either we hit a problem reading the contents of the next source
window (not unlikely, if we think this might be linked to transient disk
problems), or we hit a problem calculating the svndiff window itself.
[Though note that by window 95, the source window length has dropped to
zero, which I'm still assuming means that we ran out of source data:
it seems surprising, though not impossible, that we might get a read
error instead of an EOF during window 97.]
I'm still not sure why we restarted writing the delta-rep rather than
abort the transaction, however.
Ah: note that this probably can't be a deterministic problem with the
svndiff/vdelta code, since we repeated the operation successfully the
second time around.
Regards,
Malcolm
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sun Dec 4 17:24:26 2005