[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: revprop packing: why aren't _all_ revprops packed?

From: Daniel Shahaf <danielsh_at_elego.de>
Date: Fri, 1 Jul 2011 17:49:11 +0300

[ strictly speaking I'm not addressing /all/ your points, if I missed
any feel free to re-raise them ]

Mark Phippard wrote on Fri, Jul 01, 2011 at 09:27:57 -0400:
> On Fri, Jul 1, 2011 at 9:22 AM, Daniel Shahaf <danielsh_at_elego.de> wrote:
> > Mark Phippard wrote on Fri, Jul 01, 2011 at 09:10:31 -0400:
> >> On Thu, Jun 30, 2011 at 5:21 PM, Peter Samuelson <peter_at_p12n.org> wrote:
> >> >
> >> > [kmradke_at_rockwellcollins.com]
> >> >> I would love to have revprop packing, but not at the cost of
> >> >> potentially disabling the use of traditional backup software.
> >> >>
> >> >> Is there a way to disable fsfs revprop packing, or at least have
> >> >> it function in an atomic way like the regular rev packing?
> >> >
> >> > Hijacking the thread to veer _slightly_ off topic:
> >> >
> >> > Why is revprop packing an explicit 'svnadmin pack' operation?  If we
> >> > agree to put revprops in sqlite, why not do that from the start?  Just
> >> > open the shard-specific sqlite file, creating it if necessary, and
> >> > write the new set of revprops there.  No distinction between packed and
> >> > unpacked revprops, no 'min-unpacked-revprop' file.
> >> >
> >> > Was there a good reason not to do it that way?
> >>
> >> Wouldn't the concerns that Kevin has raised kick in if we did this?
> >> Basically you could not do a safe backup of a repository because if a
> >> commit happens during the backup we will be writing to the SQLite
> >> database.
> >>
> >
> > They would.
> >
> >
> > For the record, some ways to solve that concern are --
> >
> > 1. hotcopy
> > 2. take an atomic diks snapshot
> > 3. teach FSFS to pack revision shards without packing revprop shards
> > 4. backup just revprops.db in the SQLite-safe way
> >   (grab an SQLite-level lock, cp(1), release lock)
>
> I guess I am confused then. Especially after reading the issue you
> filed. This is how I interpret this, which maybe wrong.
>
>
> 1) Kevin raised some concerns about revprop packing and backup.
>
> 2) After some back and forth, I think the conclusion was that given
> the way it is currently implemented there is not much danger. It is
> pretty easy to do a clean backup safely.
>
> 3) Rather than all go back to our normal lives, we have decided we
> should block the release until we can make changes so that revprop
> packing and backups will now be very dangerous and all of Kevin's
> concerns are now valid.
>
> How does this make sense?
>

As I see it, there are several balls in the field:

* allow safe backups with revprops in an SQLite database
  (Kevin's original point, and the one "> >"-quoted hereinabove)

* store revprops differently
  (Ivan and Peter's suggestions)

I think some of the suggestions (in the latter category) do make sense.
They will be easier to implement them before we release f5, due to not
having to support upgrade from f5. So I filed a 1.7 blocker "Determine
how we want to refactor FSFS revprops storage, then do it".

> I thought the only proposal we were going to consider was Ivan's
> proposal to get rid of SQLite entirely and just use normal packed
> files as we do for the revisions.
>

As I said on the issue, I am interested in allowing the use of SQLite
for revprops to be optional. Especially if we're going to be
refactoring it so close to the release.

> --
> Thanks
>
> Mark Phippard
> http://markphip.blogspot.com/
Received on 2011-07-01 16:50:05 CEST

This is an archived mail posted to the Subversion Dev mailing list.