Re: New SVNSYNC bug?

On Tue, Apr 29, 2008 at 1:41 PM, David Summers
<david_at_summersoft.fay.ar.us> wrote:
>
> On Tue, 29 Apr 2008, Mark Phippard wrote:
>
>
> > On Tue, Apr 29, 2008 at 4:16 PM, David Glasser <glasser_at_davidglasser.net>
> wrote:
> >
> > >
> > > On Tue, Apr 29, 2008 at 12:11 PM, David Summers
> > > <david_at_summersoft.fay.ar.us> wrote:
> > > > On Tue, 29 Apr 2008, Lieven Govaerts wrote:
> > > >
> > > >
> > > >> Justin Erenkrantz wrote:
> > > >>
> > > >>> On Mon, Apr 28, 2008 at 3:48 PM, David Glasser
> > > > <glasser_at_davidglasser.net>
> > > >>> wrote:
> > > >>>> There were in fact changes to the svnsync RA calls as well as to
> the
> > > >>>> svnserve protocol for it, so I wouldn't be horribly surprised if
> we
> > > >>>> screwed something up in the compatibility code or something.
> > > >>>
> > > >>> FWIW, we're starting to deploy the mirroring code over at
> apache.org
> > > >>> and we're seeing a similar bug to what David S. is seeing: new
> files
> > > >>> aren't being added over replay. Changed/modified files are okay.
> > > >>>
> > > >>> (Sadly, our master SVN server has suffered a hard drive failure; so
> > > >>> this is on the back burner for us now.) -- justin
> > > >>>
> > > >>>
> > > >>
> > > >> David, Justin,
> > > >>
> > > >> For completeness sake:
> > > >> 1. between master and svnsync client you're using http, with ra_serf
> or
> > > > ra_neon?
> > > >> 2. between svnsync and slave you're using what? ra_local?
> > > >>
> > > >>
> > > >
> > > > Maybe I'm not understanding (terminology?) something, but here is my
> setup.
> > > >
> > > > 1. I commit from a WinXP SP2 client using Subversion command-line
> client
> > > > 1.4.6 with http method. (I commit to the web-dav proxy enabled slave
> but as
> > > > per your message above, that may not be relevant, the master is not
> being
> > > > corrupted as far as I can tell).
> > > >
> > > > 2. The master server receives the commit and in the post-commit hook
> does a
> > > > svnsync to the slave.
> > > >
> > > > 3. On the slave server I have svnserve running (svn method) to do the
> sync
> > > > between the master and slave.
> > > >
> > > > When I then subsequently check the slave server with "svn log -v"
> then I
> > > > see no information in "Changed Paths" and no log message.
> > > >
> > > > Hope that helps. I'm working on setting up a controlled test with
> rc3 to
> > > > see what happens and if I can narrow it down any further.
> > > >
> > > > When I backed the slave back down from rc3 to r27613 then things
> started
> > > > working fine again.
> > > >
> > > > Also, I'm beginning to wonder if the problem only showed up on a
> slave repo
> > > > created recently with rc3 or higher but I don't remember exactly when
> I
> > > > installed rc3 or created the new slave repo. I *think* I'm only
> seeing
> > > > corruption on my newest small repo and not ones created earlier with
> r27613
> > > > or older, but I can't swear to it yet.
> > >
> > > I'm not sure that this would be relevant, but there are a couple of
> > > places you are breaking our abstractions and doing things by hand:
> > >
> > >
> > >
> > > > Here is my script I use to create slave repos:
> > > > #! /bin/bash
> > > > # Set up a mirror from SAMSON
> > > >
> > > > usage() {
> > > > echo "Usage: setup.mirror RepositoryName
> > > > http://SourceHost/SourceRepository"
> > > > exit 1
> > > > }
> > > >
> > > > if [ ! -f repos/TEMPLATE/README.txt ]; then
> > > > echo "Missing TEMPLATE repository."
> > > > exit 1
> > > > fi
> > > >
> > > > if [ "$1"x = "x" ]; then
> > > > usage
> > > > fi
> > > >
> > > > if [ "$2"x = "x" ]; then
> > > > usage
> > > > fi
> > > >
> > > > REPO_NAME=$1
> > > > SOURCE_REPOSITORY=$2
> > > >
> > > > result=`echo $SOURCE_REPOSITORY | grep -i http://`
> > > >
> > > > if [ "$result"x = "x" ]; then
> > > > usage
> > > > fi
> > > >
> > > > # Create new repository
> > > > svnadmin --pre-1.5-compatible create repos/$REPO_NAME
> > > >
> > > > # Zap initial template files
> > > > rm -rf repos/$REPO_NAME/hooks/*.tmpl
> > > >
> > > > # Copy template hooks from TEMPLATE repo
> > > > cp repos/TEMPLATE/hooks/* repos/$REPO_NAME/hooks
> > > >
> > > > # Zap initial conf files.
> > > > rm -rf repos/$REPO_NAME/conf/*
> > > >
> > > > # Setup svnserve.conf
> > > > (cd repos/$REPO_NAME/conf; ln -s ../../../svnserve.conf .)
> > >
> > > What's in the conf?
> > >
> > > Specifically, could there be any authz issues here, where your
> > > svnsyncing user can't read all the files?
> > >
> > >
> > > > # Get UUID from source repository
> > > > uuid=`svn info $SOURCE_REPOSITORY | grep -i uuid | sed -e
> 's/^.*:\s//'`
> > > >
> > > > echo UUID = \"$uuid\"
> > > >
> > > > echo $uuid > repos/$REPO_NAME/db/uuid
> > >
> > > You can use svnadmin setuuid here now.
> > >
> > >
> > > > # Set up for SYNC operation
> > > > svnsync init file:///home/svnroot/repos/$REPO_NAME $SOURCE_REPOSITORY
> > > >
> > > > svn ps svn:sync-from-url svn://samson/$REPO_NAME
> > > > file:///home/svnroot/repos/$REPO_NAME -r0 --revprop --username=svnsync
> > >
> > > Why not just use the right URL with svnsync init in the first place?
> > >
> > >
> > > > chown -R apache:apache repos/$REPO_NAME
> > > >
> > > > exit 0
> > > >
> > > >
> > > > Here is my script I run on the master server to sync to the slave
> > > > (razorback) on post-commit.
> > > >
> > > > "C:\Program Files\Subversion\bin\svnsync" sync svn://razorback/%1%
> > >
> > > Hmm, so the master is Windows, and the slave is Unix? Interesting;
> > > this stuff might be undertested on Windows, though I'm not sure why it
> > > should make a difference.
> > >
> >
> > Did you ever use this with one of the 1.5 pre-release versions before
> > we nailed down the repository format? The repository internals
> > changed a lot over time and people that used the in-between stage
> > binaries could definitely get weird results.
> >
> >
> >
>
> As stated in my first email, I've used the web-dav proxy feature since
> 2006/12 within several days of when the feature was first made available, so
> my answer is "yes"; I'ved used the 1.5 pre-release versions before the
> repository format was nailed down.
>
> What is "strange" or "interesting" is that I didn't have a problem up
> through r27613 but rc3 seems to have this problem.

Again, it's not that strange, but is interesting... there have been
*significant* changes to svnsync, especially the svnserve protocol,
since r27613.

--dave

> I guess it is possible that I need to create or re-create my repo(s) with
> some new parameters or something, but what would that be? The only
> parameter I know of at the moment would be the --pre-1.5-compatible which
> I'm currently using.
>
> But I did do that before sending my first email, I re-created the
> repository with the rc3 version and tried and got the stated results.
>
> If I can reproduce the problem on a strictly 1.5.0-rc4 repo then I could
> also try doing a binary search through the intervening commits between rc3
> and r27613 and try to find out when it broke.
>
> ....Stay tuned.....
>
> Thanks!
>
> --
> David Wayne Summers "Linux: Because reboots are for hardware
> upgrades!"
> david_at_summersoft.fay.ar.us PGP Key:
> http://summersoft.fay.ar.us/~david/pgp.txt
> PGP Key fingerprint = 0B44 B118 85CC F4EC 7021 1ED4 1516 5B78 E320 2001
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_subversion.tigris.org
> For additional commands, e-mail: dev-help_at_subversion.tigris.org
>
>

-- 
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe_at_subversion.tigris.org
For additional commands, e-mail: dev-help_at_subversion.tigris.org