[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: SHA-1 collision in repository?

From: Johan Corveleyn <jcorvel_at_gmail.com>
Date: Wed, 28 Feb 2018 08:57:40 +0100

[ Please keep the users list in cc. ]

On Tue, Feb 27, 2018 at 11:38 PM, Myria <myriachan_at_gmail.com> wrote:
> On Tue, Feb 27, 2018 at 2:22 PM, Johan Corveleyn <jcorvel_at_gmail.com> wrote:
>> On Tue, Feb 27, 2018 at 10:09 PM, Myria <myriachan_at_gmail.com> wrote:
>>>
>>> On Tue, Feb 27, 2018 at 05:54 Philip Martin <philip_at_codematters.co.uk>
>>> wrote:
>>>>
>>>> Myria <myriachan_at_gmail.com> writes:
>>>>
>>>> > -bash-4.1$ sqlite3 rep-cache.db "select * from rep_cache where
>>>> > hash='db11617ef1454332336e00abc311d44bc698f3b3'"
>>>> > db11617ef1454332336e00abc311d44bc698f3b3|604440|34|134255|136680
>>>> >
>>>> > The line from the grep -a command containing that hash is below. They
>>>> > all match.
>>>> > text: 604440 34 134255 136680 c9f4fabc4d093612fece03c339401058
>>>> > db11617ef1454332336e00abc311d44bc698f3b3 604439-cyqm/_13
>>>>
>>>> The rep-cache looks correct. There doesn't seem to be any corruption in
>>>> the repository: you confirmed that you could retreive the revision in
>>>> question, and that you could verify the revision, and the rep-cache
>>>> looks OK. So why is the commit that attempts to reuse the data in the
>>>> revision failing? I don't know :-(
>>>>
>>>> > In other news, unknown whether related to the current problem, my
>>>> > attempt to clone the repository to my local computer is failing:
>>>> >
>>>> > D:\>svnsync sync file:///d:/svnclone
>>>> > Transmitting file data
>>>> >
>>>> > .....................................................................................................................................................svnsync:
>>>> > E160000: SHA1 of reps '227170 153 193 57465
>>>> > bb52be764a04d511ebb06e1889910dcf
>>>> > e6291ab119036eb783d0136afccdb3b445867364 227184-4vap/_4o' and '-1 0
>>>> > 193 57465 bb52be764a04d511ebb06e1889910dcf
>>>> > e6291ab119036eb783d0136afccdb3b445867364 227184-4vap/_4o' matches
>>>> > (e6291ab119036eb783d0136afccdb3b445867364) but contents differ
>>>> > svnsync: E160004: Filesystem is corrupt
>>>> > svnsync: E200014: Checksum mismatch while reading representation:
>>>> > expected: bb52be764a04d511ebb06e1889910dcf
>>>> > actual: 80a10d37de91cadc604ba30e379651b3
>>>> >
>>>> > This is odd, because revision 227185 (the revision it's trying to
>>>> > commit) verifies fine on the originating server:
>>>>
>>>> That's an error committing to the new repository on your local machine,
>>>> i.e. the problem is in the new repository not the repository on the
>>>> originating server. Can you run "svnadmin verify" on the new
>>>> repository? You may want to use -M to increase the cache size for the
>>>> verify command as the default is small.
>>>>
>>>> It would be odd for svnsync to create a corrupt repository, so I half
>>>> expect verify to report no problems. If that is the case it seems to be
>>>> the original pproblem again: an apparently valid repository with a
>>>> checksum error only on commit. So this problem is happening on two
>>>> repositories, on two machines with different OS.
>>>
>>>
>>> Not to mention that the two revisions complained about are unrelated, and
>>> 2/3 the repository history apart.
>>>
>>> One thing that's interesting is that the commit the svnsync failed on is a
>>> gigantic commit. It's 1.8 GB. Maybe that svnsync is failing because of a
>>> Subversion bug with huge files...?
>>>
>>> I started an svnadmin verify on my incomplete local copy last night, and no
>>> problems were reported when it finished this morning. I'll try again with
>>> this -M option you mention.
>>>
>>> I'll also start an svnsync from a Linux machine.
>>>
>>> I'm going to see how hard it would be to just copy the 43 GB repository
>>> directly. We'd have to shut down Subversion service during the copy, so it
>>> might be a while before I have a chance to.
>>
>> What version of SVN server are you using actually? AFAICS you never
>> mentioned this.
>>
>> I'm wondering whether this is related to the bug that was fixed for 1.8.x here:
>>
>> http://svn.apache.org/viewvc?view=revision&revision=1803435
>>
>> ... or a similar problem.
>> I'm actually not sure whether that bugfix was released already (it's
>> not mentioned in CHANGES).
>>
>> See also the users@ thread it references (an false positive of a SHA-1
>> collision occurred during 'svnadmin load'):
>> https://lists.apache.org/thread.html/b475d74442bdf93b21c8656ab2289b4c61e0d90efdafc8a16ddca694@%3Cusers.subversion.apache.org%3E
>>
>> OTOH there was also another report of a SHA-1 collision during
>> 'svnadmin load', this time with 1.9.7. We never got to the bottom of
>> that one either:
>> https://svn.haxx.se/users/archive-2018-01/0062.shtml
>>
> Both the server and my desktop system are on Subversion 1.9.7.
>

-- 
Johan
Received on 2018-02-28 08:58:15 CET

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.