Awesome! Thanks for looking in to this over the weekend guys. I look forward to 1.8.19.
Regards,
David
-----Original Message-----
From: Stefan Fuhrmann [mailto:stefan2_at_apache.org]
Sent: Sunday, July 30, 2017 10:14 AM
To: Bert Huijben <bert_at_qqmail.nl>; David Engel <DEngel_at_magnitude.com>; users_at_subversion.apache.org
Subject: Re: Checksum mismatch bug in 1.8.18
Hi David,
thanks for the report; the issue turned out to be quite serious.
The underlying bug has been present since at least as early as
1.8.0 but only surfaced with the added SHA1-collision detection
code in 1.8.18.
Under certain circumstances, yet-to-be-committed data would be
cached under the wrong key. The impact ranges from bogus errors
as you experienced them to silent repository corruption that only
a 'svnadmin verify' run would detect. So far, no such corruption
has been demonstrated but it remains a plausible scenario.
Luckily, the problem seems to be mostly restricted to 'svnadmin
load', with the "rep sharing" enable for the respective repo.
'svnadmin load -M 0' will also not have that issue.
I fixed the code and the fix should be shipped with the upcoming
1.8.19 release.
-- Stefan^2.
On 29.07.2017 14:37, Bert Huijben wrote:
> This specific error message was added in the last release, so yes it is new in
> the last versions. The last 1.9.x version also added it and I'm surprised that
> you see the error on 1.8 and not 1.9 (or vice versa).
>
> It tries to tell you that you have two files with an identical SHA-1 hash, but
> different contents. A case that we accidentally didn't catch before, that might
> have caused some data loss under extreme circumstances.
>
> This scenario is pretty unlikely to trigger, unless you specifically try to do
> that. I'll try to verify what exactly is in your dump file to see if you really
> hit this scenario, or that we have a bug in our code that tries to catch this
> scenario. Assuming you properly verified with 1.9.6 (and not 1.9.5 or earlier) I
> assume that this is a bug in 1.8.18, as other scenarios are really extremely
> unlikely.
>
> I'm not sure I'll have time to look into this before Monday though, so perhaps
> one of the other developers beats me to it.
>
> Bert
>
> *From:*David Engel [mailto:DEngel_at_magnitude.com]
> *Sent:* zaterdag 29 juli 2017 00:01
> *To:* users_at_subversion.apache.org
> *Subject:* Checksum mismatch bug in 1.8.18
>
> Hi,
>
> I think I'm running into a bug in svnadmin in the latest 1.8 release (1.8.18).
> It is not present in 1.8.17 nor 1.9.5 nor 1.9.6. I'm guessing it's related to
> the recent changes around strict repsharing. I'm not subscribed to the mailing
> list, so please CC me on any responses.
>
> Here are the relevant details:
>
> OS: Windows 10 and Windows 2012 R2
>
> Subversion release: 1.8.18
>
> Both SlikSVN release:
>
> svnadmin, version 1.8.18-SlikSvn-1.8.18-X64 (SlikSvn/1.8.18) X64
>
> compiled Jul 17 2017, 13:03:37 on x86_64-microsoft-windows6.2.9200
>
> And Collabnet release:
>
> svnadmin, version 1.8.18 (r1800620)
>
> compiled Jul 7 2017, 05:51:59 on x86_64-microsoft-windows6.2.9200
>
> Performing an svnadmin load results in the following error:
>
> ...
>
> <<< Started new transaction, based on original revision 650
>
> * editing path :
> Dev/Common/Utils/External/StampVerData/MetaBuilderVersion.inf ... done.
>
> * editing path :
> Dev/Common/Utils/External/StampVerData/MetaBuilderVersionDBG.inf ... done.
>
> * editing path : Dev/Common/Utils/External/StampVerData/OracleVersion.inf
> ...svnadmin: E160000: SHA1 of reps '-1 0 45 132 a6ea37d29094deece56250ebe167ce46
> 6a2d6bc4c7e58c0d2627a1b0dd5b23e542aec4d1 650-i2/_8' and '-1 136 45 132
> a6ea37d29094deece56250ebe167ce46 6a2d6bc4c7e58c0d2627a1b0dd5b23e542aec4d1
> 650-i2/_8' matches (6a2d6bc4c7e58c0d2627a1b0dd5b23e542aec4d1) but contents differ
>
> svnadmin: E160004: Filesystem is corrupt
>
> svnadmin: E200014: Checksum mismatch while reading representation:
>
> expected: a6ea37d29094deece56250ebe167ce46
>
> actual: 5f696f5d0755f3bcb5e927bdfba4bfa8
>
> In order to reproduce the error, you can use the attached VersionStamps3
> repository dump file. It was taken with the same version of svnadmin from a
> multi-GB repository that's been around for over a decade and pruned
> (svndumpfilter) to a small set of files and revisions in one folder while still
> encountering the error. I first encountered the error another place very
> quickly in an incremental load on top of a repo that was loaded from 1.8.14, but
> that one was on a revision north of 120,000+. This is way easier to reproduce.
> I tried filtering the dump further and dropping empty revisions, but the error
> did not occur in those cases. The error also did not occur using 1.8.17,
> 1.9.5., and 1.9.6. It also did not occur in 1.8.18 if I set enable-rep-sharing
> = false before performing the load.
>
> Repro.bat script:
>
> @echo off
>
> :defineCommands
>
> rem You might need to adjust these lines to point to your
>
> rem compiled-from-source Subversion binaries, if using those:
>
> for %%i in (svn.exe) do set SVN="%%~$PATH:i"
>
> for %%i in (svnadmin.exe) do set SVNADMIN="%%~$PATH:i"
>
> :defineUrls
>
> rem Only supported access method: file:// <file:///>. If http:// or svn://, then
>
> rem you'll have to configure it yourself first.
>
> set URL=file:///%CD%/repos
>
> set URL=%URL:\=/%
>
> echo Base url for repo: %URL%
>
> :cleanAllDirsAndCreateRepo
>
> if exist repos rmdir /s /q repos
>
> if exist import-me rmdir /s /q import-me
>
> if exist wc rmdir /s /q wc
>
> %SVNADMIN% create repos
>
> pause
>
> :prepareGreekTree
>
> echo Making a Tree for import...
>
> mkdir import-me\Dev\Common\Utils\External
>
> echo Importing it...
>
> cd import-me
>
> %SVN% import -q -m "Initial import." %URL%
>
> cd ..
>
> rem This is where your reproduction recipe goes.
>
> svnadmin --version
>
> svnadmin load repos < VersionStamps3
>
> goto :eof
>
> Thanks,
>
> David
>
Received on 2017-07-31 17:11:35 CEST