Philip Martin wrote on Thu, Jul 07, 2011 at 08:36:06 +0100:
> Daniel Shahaf <danielsh_at_elego.de> writes:
>
> > The process to edit a revprop that had been packed would be:
> >
> > * grab write lock
> > * prepare a new pack-file-with-inline-manifest
> > * move-into-place, atomically replacing the previous pack file
> > * ungrab write lock
> >
> > That is what guarantees cp(1) consistency that Hyrum mentions.
>
> Atomic replace is going to involve a retry loop on Windows, and so could
> potentially take a long time. Virus scanners on repository disks?
>
Yes, this is going to be a problem.
> Holding the write lock will block readers so for better concurrency we
> want to minimise the time that a write lock is held, or have more locks.
>
> Is there one lock per pack file or one lock per repository? Having
> multiple locks means that a write only blocks a proportion of the reads,
> but also means an operation like log has to acquire multiple locks (in
> series not in parallel).
>
It will block writers, not readers.
But having more fine-grained locks seems a good idea.
> We can move the prepare/write outside the write lock by using a sequence
> number in the pack file. Then the algorithm could be:
>
> * prepare a new pack file with incremented sequence number
> * grab write lock
> * check old sequence number
> * if changed: drop lock and start again
> * otherwise: atomic replace pack file
> * drop write lock
>
> That doesn't help with the retry problem on Windows, but may reduce the
> time the write lock is held, particularly if the pack file is large. It
> shoud work well if writes are much less common than reads. If writes
> are common then the simple write lock serialisation may be better.
>
Good idea.
> The two algorithms should interoperate so, even if we use a simple write
> lock today, implementing the sequence number would give us options in
> the future.
Received on 2011-07-07 15:40:49 CEST