You are making things very hard for yourself by running 1.6.11, it is
very old and no longer maintained, unless your distribution is
backporting fixes. A newer version will have fewer bugs and better
performance.
1.6 has a hotcopy bug
http://subversion.tigris.org/issues/show_bug.cgi?id=3596
the effects are hard to predict but one possibility is corrupt revisions
when committing to the hotcopy. You could delete the file
db/rep-cache.db from the hotcopy and see if this fixes the problem. In
1.8 the hotcopy bug is fixed, 1.8 also attempts to detect and avoid
problems caused by a corrupt rep-cache.db.
Tony Sweeney <sweeney_at_addr.com> writes:
> Hi,
>
> Mise en scene:
>
> We have three Subversion servers, version 1.6.11 running on
> CentOS6. One is the live master and and the others are hot standbys in
> different datacenters. The intent is that these are kept in sync with
> the master using svnsync from a post-commit hook. One of the slaves
> is already set up this way and has been since it was deployed by one
> of my predecessors at this company. This is syncing nicely. The
> other slave instance was also originally set up this way but was then
> modified to instead use a block level filesystem mirroring technology,
> again by one
> of my predecessors. We have since concluded that the
> mirroringsolution was not sufficiently trustworthy, and so we would
> like to put it back to the svnsync method that it had run in the past.
> The live master has in excess of 366,000 revisions and new revisions
> come in at ~500 commits/day.
>
> One solution would be to simply create an empty repo, set the
> necessary r0 revprops and hook scripts and sync from scratch.
> However, we estimate that this would take 10-15 days due to the size
> of the repository, so we would like to short-circuit this by starting
> from a hotcopy. We take hotcopies nightly, save them to a backup
> server and 'svnadmin verify' them there. I am trying to bring this
> slave back on sync using the method described here:
>
> http://www.cardinalpath.com/how-to-use-svnsync-to-create-a-mirror-backup-of-your-subversion-repository/
>
> But using a hotcopy rather than a dump.
>
> The master hotcopy is restored in the correct location on the slave
> and the hook scripts fixed up using Puppet (master and slave
> necessarily have different hook script setups). Permissions are fixed
> up to reflect that the hotcopy is taken as root but the server is
> running under Apache. I set the svn:sync-last-merged-rev revprop on
> r0 to reflect the newest revision on the hotcopy. At this point (or
> so I thought) I should be able to replicate over the changes since
> last night by running svnsync manually and then enable it in the hook
> script.
>
> The problem is that that the revisions seem to be corrupted on
> transfer to this slave.
>
> From the master:
>
> [root_at_uk-svn-1-new yoyodyne]# /usr/bin/svnsync --non-interactive --trust-server-
> cert sync https://uk-svn-2-new.uk.yoyodyne.lan/svn/yoyodyne
> --sync-username svnsync --sync-password XXXXXXXX --source-username
> svnsync --source-password XXXXXXXX --config-dir
> /repos/svn/yoyodyne/hooks/certs
> Transmitting file data ..
> Committed revision 367219.
> Copied properties for revision 367219.
> Transmitting file data .
> Committed revision 367220.
> Copied properties for revision 367220.
> Transmitting file data .
> Committed revision 367221.
> Copied properties for revision 367221.
> Transmitting file data .svnsync: Corrupt representation '367220 0 41 29949930e5
> 5f203ff7027917e765e6ca7d 2654a46e5fff1f98b747a77ebf600c10c895fbcc367219-7wnu/_6'
> You have new mail in /var/spool/mail/root
> [root_at_uk-svn-1-new yoyodyne]#
>
> In fact, checking on the slave, they have all come across corrupt:
>
> [root_at_uk-svn-2-new svn]# svnadmin verify -r 367200:HEAD /repos/svn/yoyodyne
> * Verified revision 367200.
> * Verified revision 367201.
> * Verified revision 367202.
> * Verified revision 367203.
> * Verified revision 367204.
> * Verified revision 367205.
> * Verified revision 367206.
> * Verified revision 367207.
> * Verified revision 367208.
> * Verified revision 367209.
> * Verified revision 367210.
> * Verified revision 367211.
> * Verified revision 367212.
> * Verified revision 367213.
> * Verified revision 367214.
> * Verified revision 367215.
> * Verified revision 367216.
> * Verified revision 367217.
> * Verified revision 367218.
> svnadmin: Corrupt representation '367219 0 50
> 633695f7c627c997b8b4065db2d4cb4db53c
> 1f15b1d8cea246d731676f43b2df4c74a1710793 367218-7wnt/_d'
> svnadmin: Malformed representation header
> You have mail in /var/spool/mail/root
> [root_at_uk-svn-2-new svn]#
>
> Both the master and working slave verify all revisions clean at all
> times. I've tried multiple hotcopies and every time the first
> revision that it tries to sync comes across as corrupt.
>
> On the master the revision file looks as follows:
>
> [root_at_uk-svn-1-new yoyodyne]# head -30 db/revs/367/367219
> DELTA 366873 14907 1619
> SVN??@
>
> ?=??h?&ng serialVersionUID = 1;
> ENDREP
> DELTA 366873 16539 1063
> SVN?J?7?V?)|???"?Y?? ??private boolean hasMoreResults = trueif(results.size() <
> requiredResults){
>
> hasMoreResults = false;
> }
> hasMoreResults;
> }
> }
> ENDREP
> id: 1j-366873.0-228442.r367219/295
> type: file
> pred: 1j-366873.0-228442.r366873/30632
> count: 1
> text: 367219 81 183 2999 2880cca7c6c56dea580d5c212595846c dc0cc36f718b468fc08817
> 88900f096d5cd4913f 367218-7wnq/_e
> props: 366873 30580 39 0 f2ea0bdf4310dae3aef66de358bcb0e1
> cpath:
> /MIME/MADb/trunk/server/src/main/java/com/yoyodyne/madb/query/memory/DBServiceQueryThread.java
> copyroot: 228442 /MIME/MADb
>
> id: 1h-366873.0-228442.r367219/692
> type: file
> pred: 1h-366873.0-228442.r366873/32681
> count: 1
> text: 367219 0 50 6336 95f7c627c997b8b4065db2d4cb4db53c 1f15b1d8cea246d731676f43
> b2df4c74a1710793 367218-7wnq/_d
> props: 366873 32629 39 0 f2ea0bdf4310dae3aef66de358bcb0e1
> cpath:
> /MIME/MADb/trunk/server/src/main/java/com/yoyodyne/madb/query/memory/CachedRowList.java
> copyroot: 228442 /MIME/MADb
>
> [root_at_uk-svn-1-new yoyodyne]#
>
> The working slave looks as follows:
>
> [root_at_us-svn-1 yoyodyne]# head -30 db/revs/367/367219
> DELTA 366873 14907 1619
> SVN??@
>
> ?=??h?&ng serialVersionUID = 1;
> ENDREP
> DELTA 366873 16539 1063
> SVN?J?7?V?)|???"?Y?? ??private boolean hasMoreResults = trueif(results.size() <
> requiredResults){
>
> hasMoreResults = false;
> }
> hasMoreResults;
> }
> }
> ENDREP
> id: 1j-366873.0-228442.r367219/295
> type: file
> pred: 1j-366873.0-228442.r366873/30631
> count: 1
> text: 367219 81 183 2999 2880cca7c6c56dea580d5c212595846c dc0cc36f718b468fc08817
> 88900f096d5cd4913f 367218-7yyk/_e
> props: 366873 30579 39 0 f2ea0bdf4310dae3aef66de358bcb0e1
> cpath:
> /MIME/MADb/trunk/server/src/main/java/com/yoyodyne/madb/query/memory/DBServiceQueryThread.java
> copyroot: 228442 /MIME/MADb
>
> id: 1h-366873.0-228442.r367219/692
> type: file
> pred: 1h-366873.0-228442.r366873/32680
> count: 1
> text: 367219 0 50 6336 95f7c627c997b8b4065db2d4cb4db53c 1f15b1d8cea246d731676f43
> b2df4c74a1710793 367218-7yyk/_d
> props: 366873 32628 39 0 f2ea0bdf4310dae3aef66de358bcb0e1
> cpath: /MIME/MADb/trunk/server/src/main/java/com/yoyodyne/madb/query/memory/Cach
> edRowList.java
> copyroot: 228442 /MIME/MADb
>
> [root_at_us-svn-1 yoyodyne]#
>
> But on the failing slave:
> [root_at_uk-svn-2-new yoyodyne]# head -30 db/revs/367/367219
> id: 1j-366873.0-228442.r367219/0
> type: file
> pred: 1j-366873.0-228442.r366873/30632
> count: 1
> text: 367219 81 183 2999 2880cca7c6c56dea580d5c212595846c dc0cc36f718b468fc08817
> 88900f096d5cd4913f 367218-7wnt/_e
> props: 366873 30580 39 0 f2ea0bdf4310dae3aef66de358bcb0e1
> cpath:
> /MIME/MADb/trunk/server/src/main/java/com/yoyodyne/madb/query/memory/DBServiceQueryThread.java
> copyroot: 228442 /MIME/MADb
>
> id: 1h-366873.0-228442.r367219/395
> type: file
> pred: 1h-366873.0-228442.r366873/32681
> count: 1
> text: 367219 0 50 6336 95f7c627c997b8b4065db2d4cb4db53c 1f15b1d8cea246d731676f43
> b2df4c74a1710793 367218-7wnt/_d
> props: 366873 32629 39 0 f2ea0bdf4310dae3aef66de358bcb0e1
> cpath:
> /MIME/MADb/trunk/server/src/main/java/com/yoyodyne/madb/query/memory/CachedRowList.java
> copyroot: 228442 /MIME/MADb
>
> PLAIN
> K 19
> CSVQueryThread.java
> V 37
> file 1e-366873.0-228442.r366873/31862
> K 18
> CachedRowList.java
> V 35
> file 1h-366873.0-228442.r367219/395
> K 25
> DBServiceQueryThread.java
> V 33
> [root_at_uk-svn-2-new yoyodyne]#
>
> Each time I've tries this the failure mode appears to be the same.
> The initial 'DELTA <E2><80><A6> ENDREP' sections are missing from the
> copied over file, it starts with the 'id:' line directory. Is there
> anything I can do to debug this further?
>
> Tony.
>
--
Philip Martin | Subversion Committer
WANdisco // *Non-Stop Data*
Received on 2015-02-20 11:21:53 CET