[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Repository migrated via SVNSYNC is much smaller than one migrated using SVNADMIN DUMP/LOAD : SOLVED

From: Christopher Lamb <christopher.lamb_at_ch.ibm.com>
Date: Fri, 29 Aug 2014 09:38:10 +0200

Hi All

Thanks to Andreas and Les for your kind hints.

Having read and re-read your replies, and reviewing how I was calling
SVNSYNC, and even a bit of RTFM, I have now solved this issue.

The Eureka moment came when I looked at the output from SVNSYNC, and
compared it to that shown in the SVNSYNC documentation.

It turns out I was incorrectly addressing the source repository. (not
exactly Les's typo, but the result is the same)

Strangely what I was doing was correct enough for properties and revisions
to be replicated, but not correct enough to transmit files .....

Lets call my original attempt(s) "Bad", and the corrected approach "Good".

"Bad" Calls:
svnsync init svn://target_server/target_repo svn://source_server
/source_repo
svnsync sync svn://target_server/target_repo svn://source_server
/source_repo

"Bad" Output
....
Copied properties for revision 15.
Committed revision 16.
Copied properties for revision 16.
Committed revision 17.
Copied properties for revision 17.
Committed revision 18.
....

This certainly appeared to be doing something, and populated the revs and
revprops directories with the correct number of files. While the revprops
dir looked good, the revs dir was far too small (and of almost equal size
to the revprops dir)

I should be calling SVNSYNC as shown below

"Good" Calls:
svnsync init svn://target_server/target_repo svn://source_server
svnsync sync svn://target_server/target_repo svn://source_server

"Good" Output:
....
Transmitting file data .
Committed revision 7088.
Copied properties for revision 7088.
Transmitting file data .
Committed revision 7089.
Copied properties for revision 7089.
....

The difference is shown in the output: Here we additionally have
"Transmitting file data."

In the "bad" case I was addressing the source repo as
svn://source_server/source_repo
In the "good" case I address the source repo as
svn://source_server
In both cases the target repo is addressed as
svn://target_server/target_repo

So why the difference / confusion?

SvnServe on the target server is setup with one port (3690) for all repos,
so the repos need to be addressed as svn://target_server/target_repo

SvnServe on the source server is setup with one port per repo. As the repo
I am testing with uses the default port 3690, all I need to address it is
the sever name: svn://source_server. Presumably for other repos I will need
to add the port e.g. svn://source_server:3691

The annoying thing is that my addressing was incorrect, yet still correct
enough to so something...

Cheers

Chris

p.s
Now that I have removed CR LFs from over 600 revprops, I can use SVNADMIN
DUMP / LOAD for the actual migration, and SVNSYNC as part of the backup
plan.

From: Andreas Stieger <andreas.stieger_at_gmx.de>
To: Christopher Lamb/Switzerland/IBM_at_IBMCH,
Cc: users_at_subversion.apache.org
Date: 28.08.2014 19:38
Subject: Re: Repository migrated via SVNSYNC is much smaller than one
            migrated using SVNADMIN DUMP/LOAD

Hello,

On 28/08/14 17:12, Christopher Lamb wrote:
> While experimenting how best to do this I have tried using both SVNADMIN
> DUMP / LOAD, SVNSYNC, and even SVNRDUMP. The resulting target repos have
> dramatically different sizes, hence this mail.
>
>
> Original repo size 5.51 GB
> Almost 19,000 revisions
> SVN version 1.5.1
> FSFS
>
> Target repo(s)
> SVN version 1.7.14 (r1542130)
> FSFS
> [10.8GiB with dump load, 149 MiB with svnsync]

1. Representation sharing in 1.6:
https://subversion.apache.org/docs/release-notes/1.6.html#filesystem-improvements

2. When using svnsync, make sure the reading user has fully recursive
read access, and no subtree has restrictive controls.
From
http://svnbook.red-bean.com/en/1.7/svn.reposadmin.maint.html#svn.reposadmin.maint.replication.svnsync-partial

> svnsync isn't limited to full copies of everything which lives in a
repository. It can handle various shades of partial replication, too. For
example, while it isn't very commonplace to do so, svnsync does gracefully
mirror repositories in which the user as whom it authenticates has only
partial read access. It simply copies only the bits of the repository that
it is permitted to see. Obviously, such a mirror is not useful as a backup
solution.

Obviously in your case it may sync partially (without message) and thus
result in a smaller repository. You will get an initial hint by printing
the full HEAD tree of the root of each repository. Any differences there
would point to a problem.

As you get two different size, both are likely to occur, while you only
want #1 for a migration.

Regards,
Andreas
Received on 2014-08-29 09:38:45 CEST

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.