Philip Martin wrote on Thu, Jul 07, 2011 at 14:54:10 +0100:
> Daniel Shahaf <danielsh_at_elego.de> writes:
>
> > Hyrum K Wright wrote on Thu, Jul 07, 2011 at 08:29:31 -0500:
> >> On Thu, Jul 7, 2011 at 2:36 AM, Philip Martin
> >> <philip.martin_at_wandisco.com> wrote:
> >> >
> >> > r0 revprops are a concern, they can have different access patterns. For
> >> > example a master/slave setup running svnsync once per revision (a common
> >> > setup) will write the r0 pack file several times per revision. We don't
> >> > want the pack file to become the dominant IO.
> >> >
> >> > I wonder if we could offset the shard boundaries, so that r0 is the last
> >> > revision in the first shard and r1-rN is the second shard. Then r0
> >> > would be a shard on its own and the r0 pack file would be much smaller.
> >> > We would have to repack the repository on upgrade but the code changes
> >> > for this could be small, just +/-1 in a few places.
> >>
> >> Changing the offsets for all the shard boundaries is just asking for
> >> trouble, as it would mean that revision shards and revprop shards
> >> wouldn't match up. That seems like a pretty big "gotcha".
>
> I assumed we would "repack the repository on upgrade" and change the
> revision shards as well. Maybe that's even worse :)
>
It means that all the '% ffd->max_files_per_dir' computations need to
get an off-by-one iff they are for revprops. Special-casing r0 and the
first shard's manifest seems to me, too, to be preferable.
> >> Instead, we could just special-case r0 revprops so that they never get
> >> packed, and that the 0th shard includes r1:rN, instead of r0:rN. That
> >> would only impact the 0th shard, rather than every revprop shard.
> >
> > +1
>
> If that's easy it would be even better.
>
> Subversion log messages average about 400 characters, so if we take that
> as standard a pack file starts at about half a MB. Something like
Actually it's 499931 bytes (for the last full shard from our history).
Nice estimate :-)
> 'svnsync sync' would do about 2.5MB write IO per invocation just setting
> r0 revprops.
Received on 2011-07-07 16:14:27 CEST