On Sun, Jul 8, 2012 at 11:10 AM, Bert Huijben <bert_at_qqmail.nl> wrote:
> Hi Stefan,****
>
> ** **
>
> Why do you use a default of 64k for a revprop pack?
>
In the Apache repo, the average revprop list is <250 bytes / revision.
So, 64k is already large enough to reduce the number of files significantly.
OTOH, in many storage backends, 64k are the smallest amount of
data you can reasonably read (e.g. RAID stripes use 64k or 128k
blocks per disk).
> ****
>
> Isn’t something like 1MB a more sensible default? I would guess that for
> the 1000 revisions we usually pack that would fit in a single pack, while
> with 64K you only can get a few tens/hundreds revisions in a pack,
> depending on how large the log messages usually are.****
>
> ** **
>
> Does such a small pack size have specific benefits?
>
The small default block size minimizes the I/O and processing
overhead when accessing a single revprop: It is about twice as
expensive (manifest + pack file) as reading a non-packed revprop.
If you know that you rarely write to packed revprops *and* if you
enable revprop caching, you should enable compression for
packed revprops. Typical compression rate is 1:4 and the
default pack size limit is raised to 256k. You may choose an
even larger value.
The key is that you need revprop caching in that case or checkouts,
list -v and logs will become very slow.
-- Stefan^2.
> *From:* Stefan Fuhrmann [mailto:stefan.fuhrmann_at_wandisco.com]
> *Sent:* vrijdag 6 juli 2012 10:32
> *To:* Subversion Development
> *Subject:* Revprop packing implemented****
>
> ** **
>
> Hi devs,
>
> This week I had one of my "how hard can it be?" moments
> and finally implemented revprop packing (did that mainly
> offline). It passes all tests and seems to work pretty well.
>
> It's design deviates from the existing revprop packing branch
> in that it is more scalable and simpler to implement. Key
> differences:
>
> * Have a configurable limit (default 64k) to the pack file size.
> Concatenate revprops only up to that limit, but at least one
> revision's revprops per file. Have the manifest map revs to
> revprop pack files. IOW, let the OS do all the heavy lifting
> of storage management.
> * Make revprop packing a mandatory part of FSFS packing
> in format 6 repos. There is no need to track revprop packing
> separately nor to give it a separate format info.
>
> Since the new code will not be used unless you create
> a format 6 repo, I'd like to commit everything directly to
> /trunk for review instead of creating a new branch or
> "overwriting" the existing one.
>
> This is the order in which I want to commit the changes:
>
> * refactor existing code
> * update the design file
> * add the revprop pack support
> * add tests; write more tests
> * bump the FSFS format
>
> Any objections?
>
> -- Stefan^2.
>
> --
> Certified & Supported Apache Subversion Downloads:
> http://www.wandisco.com/subversion/download****
>
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Received on 2012-07-08 20:27:03 CEST