On Tue, Jun 16, 2015 at 11:55 PM, Evgeny Kotkov <evgeny.kotkov_at_visualsvn.com
> wrote:
> Stefan Fuhrmann <stefan.fuhrmann_at_wandisco.com> writes:
>
> > That makes we wonder how we prevent packing in old format repositories.
> > We know that hotcopying while packing can corrupt the destination repo.
>
> [...]
>
> > I'm -1 on reverting: Not being able to use hotcopy on an r/o repo is less
> > problematic than ending up with a corrupted copy.
>
> [...]
>
> > Once we are at it, let's detect pack operations during hotcopy (compare
> > pack status before & after the hotcopy). Then we can at least warn the
> > user of older format repos that their hotcopy is possibly corrupt now.
>
> [...]
>
> Could you please elaborate on this? What are the exact circumstances under
> which a concurrent 'svnadmin pack' for a hotcopy source makes a corrupted
> hotcopy destination feasible?
>
> A concurrent pack can result in the 'svnadmin hotcopy' erroring out with an
> ENOENT — in a situation when we attempt to copy a revision file that moved
> into a pack and was blown away in the cleanup happening in
> synced_pack_shard().
> But even in this case the hotcopy destination is not *corrupt*, it is just
> incomplete due to how hotcopy currently checkpoints changes in db/current
> and
> in db/min-unpacked-rev (checkpointing occurs once per shard). The end user
> is going to have a quite visible error in this case, and calling 'svnadmin
> hotcopy --incremental' after receiving an error is going to copy the
> remaining
> part of the source repository and bring the target up-to-date — just as if
> nothing happened.
>
Hi Evgeny,
Sorry, my bad, I was wrong there. I only remembered that we had and still
have the pack/hotcopy race condition in 1.8 and then I saw the recursive
directory copy calls in the /trunk hotcopy code and had my false positive.
You are absolutely right. We already copy revisions explicitly and will
simply stop upon missing files w/o bumping 'current' (as I had suggested
yesterday for a future improvement). And there is plenty of commentary
on that issue in the code as well. A user might later retry the hotcopy
and they will likely succeed.
So, at the moment, the pack lock prevents those (in some setups rare)
ENOENT errors. I think that apparent reliability (users don't see failures
once on a blue moon) is worth keeping. I'll try to come up with a patch
for the r/o issue tomorrow.
Furthermore, we should mention the ENOENT error condition in our
release notes.
> I believe that if there are known circumstances that can provoke a
> corruption
> of the target repository during hotcopy, we should fix them, but I am
> currently
> unaware of these conditions. What are they?
>
The only outstanding consistency issue with hotcopying is the locks
handling: this is the code that does plain tree copies. But I have no
idea whether that is an actual problem and whether we can do anything
about it.
Since it is something I'm currently working on in FSX, fsync'ing the copy
might be a good idea as well. In particular for people that copy to some
portable device. Nothing urgent, though.
Again, sorry for the misread!
-- Stefan^2.
Received on 2015-06-17 22:10:30 CEST