Thank you Jon, for your explanation and workaround.
Are there any "best practices" that we can advise our dev groups to follow to avoid this problem?
Otherwise, your suggestions seem to indicate I would have to run the sync on a cronjob and not with the hook script. That is something we would like to avoid. So I have added a start time comparison and sleep in a start-commit hook instead. Do you see any reason why this would cause other problems?
I am assuming that if the commits must start at least one second apart - then the sync from the post-commit hook would not be able to reach a race condition. Is this a reasonable assumption?
#!/usr/bin/sh
# START-COMMIT HOOK
# kanderse Jan 13, 2010
# The start-commit hook is invoked before a Subversion txn is created
# in the process of doing a commit.
# This script checks the start time and compares with the start time
# of the previous commit. It will cause a commit to wait one second if
# the last commit was started less than one second earlier.
# The purpose of this wait is to prevent known issue 3546 [1][2].
# a race condition involving multiple sync processes running at
# the same time that result in a corrupted proxy.
REPOS="$1"
USER="$2"
DATE1=`cat /$REPOS/hooks/start-time.txt` # previous start time
DATE2=`/usr/local/bin/date +%s` # record current start time
echo $DATE2 > /$REPOS/hooks/start-time.txt
# echo $DATE2 $DATE1 `expr $DATE2 - $DATE1`
if [ `expr $DATE2 - $DATE1` -lt 1 ]
then sleep 1 # to prevent sync race that results in sync duplication and corrupted proxy
fi
# All checks passed, so allow the commit.
exit 0
Krista Andersen
Global Development Infrastructure
Investment Technology Group, Inc.
400 Corporate Pointe, 8th Floor
Culver City, CA 90230
Direct: 213.270.7570
-----Original Message-----
From: Jon Foster [mailto:Jon.Foster_at_cabot.co.uk]
Sent: Wednesday, January 13, 2010 5:13 AM
To: Andersen, Krista; users_at_subversion.apache.org
Cc: ssi-svn_admin
Subject: RE: sync bug -> corrupted proxy repo
Hi,
Andersen, Krista [mailto:Krista.Andersen_at_itg.com] wrote:
> Twice I have seen one of my proxy repositories become corrupted due
> to an apparent bug in the svnsync sync process. Has anyone else
> seen this type of behavior from Subversion?
This is probably caused by issue 3546 [1][2]. This is a race
condition - if you have several sync processes running at the same
time then the mirror can get corrupted. You had three commits which
were 1 second apart, so your hook script started 3 copies of svnsync
within 2 seconds. I think this is the first practical report of this
bug; previous discussion was theoretical.
> Here is a comparison the output of the svn log -v for the offending
> revisions (324,325) on both the corrupted and non-corrupted proxy
> repo.
It looks like rev 323 got mirrored twice (as mirror revs 323 and 324),
then rev 324 was mirrored (as mirror rev 325).
> I am a bit concerned about the stability of Subversion since this
> is the second time in two months that I have had to fix this issue.
> Is there a patch or something to prevent this in the future?
Suggested workaround: Change your hook scripts to use the lockf or
lockfile tools[3] to ensure that only one instance of svnsync runs
at once.
Kind regards,
Jon
[1]
http://mail-archives.apache.org/mod_mbox/subversion-dev/200911.mbox/%3C2
0091127115356.GC9302_at_jack.stsp.name%3E
[2] http://subversion.tigris.org/issues/show_bug.cgi?id=3546
[3]
http://mail-archives.apache.org/mod_mbox/subversion-dev/200911.mbox/%3C2
0091127132659.GE9302_at_jack.stsp.name%3E
**********************************************************************
This email and its attachments may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Cabot Communications Ltd.
If you are not the intended recipient of this email and its attachments, you must take no action based upon them, nor must you copy or show them to anyone.
Cabot Communications Limited
Verona House, Filwood Road, Bristol BS16 3RY, UK
+44 (0) 1179584232
Co. Registered in England number 02817269
Please contact the sender if you believe you have received this email in error.
**********************************************************************
______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
This message is for the named person's use only. This communication is for
informational purposes only and has been obtained from sources believed to
be reliable, but it is not necessarily complete and its accuracy cannot be
guaranteed. It is not intended as an offer or solicitation for the purchase
or sale of any financial instrument or as an official confirmation of any
transaction. Moreover, this material should not be construed to contain any
recommendation regarding, or opinion concerning, any security. It may
contain confidential, proprietary or legally privileged information. No
confidentiality or privilege is waived or lost by any mistransmission. If
you receive this message in error, please immediately delete it and all
copies of it from your system, destroy any hard copies of it and notify the
sender. You must not, directly or indirectly, use, disclose, distribute,
print, or copy any part of this message if you are not the intended
recipient. Any views expressed in this message are those of the individual
sender, except where the message states otherwise and the sender is
authorized to state them to be the views of any such entity.
Securities products and services provided to Canadian investors are offered
by ITG Canada Corp. (member CIPF and IIROC - Investment Industry Regulatory
Organization of Canada), an affiliate of Investment
Technology Group, Inc.
ITG Inc. and/or its affiliates reserves the right to monitor and archive
all electronic communications through its network.
ITG Inc. Member FINRA, SIPC
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
Received on 2010-01-15 23:29:28 CET