[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: do away with db/revprops/*/, and a question about 'upgrade' concurrency (was: Re: Does fsfs revprop packing no longer allow usage of traditional backup software?)

From: Daniel Shahaf <danielsh_at_elego.de>
Date: Fri, 1 Jul 2011 21:30:15 +0300

Daniel Shahaf wrote on Fri, Jul 01, 2011 at 19:29:50 +0300:
> Hyrum K Wright wrote on Fri, Jul 01, 2011 at 11:21:58 -0500:
> > On Fri, Jul 1, 2011 at 8:05 AM, Daniel Shahaf <danielsh_at_elego.de> wrote:
> > > Hyrum K Wright wrote on Thu, Jun 30, 2011 at 16:33:16 -0500:
> > >> On Thu, Jun 30, 2011 at 3:27 PM, Peter Samuelson <peter_at_p12n.org> wrote:
> > >> >
> > >> > [Ivan Zhakov]
> > >> >> It should be easy to implement editing revprops without using SQLite:
> > >> >> in case someone modify revprop non-packed revprop file is created, in
> > >> >> read operation non-packed revprop file should be considered as more
> > >> >> up-to-date. In next svnadmin pack operation these non-packed files
> > >> >> should be merged back to packed one.
> > >> >
> > >> > +1.  This would basically mean there's only _one_ code path for writing
> > >> > revprops, yes?  'svnadmin pack' gets a little more complex, but the
> > >> > rest of libsvn_fs_fs gets simpler.
> > >> >
> > >> > Anyone have time to actually do this?  Converting the packed format
> > >> > from sqlite to the same format used for packed revs would be a bonus.
> > >>
> > >> I like this idea, but it would seem to introduce an additional stat()
> > >> call* for every attempt to fetch a revprop, because you'd first have
> > >> to check the "old" location, and then the packed one.
> > >
> > > I don't like the idea of writing revprops outside the DB and moving them
> > > back in. (I think I already said why elsethread?)
> >
> > (I will assume "the DB" means "the SQLite-backed revprop database".)
> >
> > I agree with you, but I don't think that's what the proposal was
> > about. My understanding would be that we'd pack revprops just like we
> > pack revision (one single concat'd file per shard), and then store any
> > edits in separate files. We'd then "repack" the edits into the pack
> > files when an admin subsequently runs 'svnadmin pack'.
>
> Yes, that's exactly what I don't like :-)

I take that back --- what Hyrum described here is not the same as my
previous understanding.

How about storing just a single serialized hash per shard, but
nameprefix the properties?

K 20
svn:fsfs:r1:svn:date
V 22
2011-07-01T18:19:54.259712Z
K 20
svn:fsfs:r0:svn:date
V 27
2011-07-01T18:19:54.259711Z
K
svn:fsfs:r1:svn:author
V 8
danielsh
END

or

K 2
r0
V 47
K 20
svn:date
V 27
2011-07-01T18:19:54.259711Z
K 2
r1
V 76
K 10
svn:author
V 8
danielsh
K 20
svn:date
V 27
2011-07-01T18:19:54.259712Z
END

or

(same as the last suggestion, but store the keys as unserialized
foo-endian uint64 per Peter's suggestion)

This doesn't require a manifest file. A propchange would read the
properties using a variant of svn_hash_read() that only includes keys
related to the given revision, and write the modifications by

* grabbing the FSFS write lock
* dumping the K-V hunk(s) for the affected revision to db/revprops/0.tmp
* concatenating the remainder of db/revprops/0 into 0.tmp,
  excluding hunks pertaining to the revision just affected.

ie, move a bit of the heavy lifting to the hash writing/reading code.

Hmm. I'm not sure yet myself that I like this idea. But I'll throw it
out there anyway :)
Received on 2011-07-01 20:31:29 CEST

This is an archived mail posted to the Subversion Dev mailing list.