FYI - forwarding from users@
Stefan Fuhrmann wrote on Sun, 30 Jul 2017 19:14 +0200:
> Hi David,
>
> thanks for the report; the issue turned out to be quite serious.
> The underlying bug has been present since at least as early as
> 1.8.0 but only surfaced with the added SHA1-collision detection
> code in 1.8.18.
>
> Under certain circumstances, yet-to-be-committed data would be
> cached under the wrong key. The impact ranges from bogus errors
> as you experienced them to silent repository corruption that only
> a 'svnadmin verify' run would detect. So far, no such corruption
> has been demonstrated but it remains a plausible scenario.
>
> Luckily, the problem seems to be mostly restricted to 'svnadmin
> load', with the "rep sharing" enable for the respective repo.
> 'svnadmin load -M 0' will also not have that issue.
>
> I fixed the code and the fix should be shipped with the upcoming
> 1.8.19 release.
>
> -- Stefan^2.
>
> On 29.07.2017 14:37, Bert Huijben wrote:
> > This specific error message was added in the last release, so yes it is new in
> > the last versions. The last 1.9.x version also added it and I’m surprised that
> > you see the error on 1.8 and not 1.9 (or vice versa).
> >
> > It tries to tell you that you have two files with an identical SHA-1 hash, but
> > different contents. A case that we accidentally didn’t catch before, that might
> > have caused some data loss under extreme circumstances.
> >
> > This scenario is pretty unlikely to trigger, unless you specifically try to do
> > that. I’ll try to verify what exactly is in your dump file to see if you really
> > hit this scenario, or that we have a bug in our code that tries to catch this
> > scenario. Assuming you properly verified with 1.9.6 (and not 1.9.5 or earlier) I
> > assume that this is a bug in 1.8.18, as other scenarios are really extremely
> > unlikely.
> >
> > I’m not sure I’ll have time to look into this before Monday though, so perhaps
> > one of the other developers beats me to it.
> >
> > Bert
> >
> > *From:*David Engel [mailto:DEngel_at_magnitude.com]
> > *Sent:* zaterdag 29 juli 2017 00:01
> > *To:* users_at_subversion.apache.org
> > *Subject:* Checksum mismatch bug in 1.8.18
> >
> > Hi,
> >
> > I think I’m running into a bug in svnadmin in the latest 1.8 release (1.8.18).
> > It is not present in 1.8.17 nor 1.9.5 nor 1.9.6. I’m guessing it’s related to
> > the recent changes around strict repsharing. I’m not subscribed to the mailing
> > list, so please CC me on any responses.
> >
> > Here are the relevant details:
> >
> > OS: Windows 10 and Windows 2012 R2
> >
> > Subversion release: 1.8.18
> >
> > Both SlikSVN release:
> >
> > svnadmin, version 1.8.18-SlikSvn-1.8.18-X64 (SlikSvn/1.8.18) X64
> >
> > compiled Jul 17 2017, 13:03:37 on x86_64-microsoft-windows6.2.9200
> >
> > And Collabnet release:
> >
> > svnadmin, version 1.8.18 (r1800620)
> >
> > compiled Jul 7 2017, 05:51:59 on x86_64-microsoft-windows6.2.9200
> >
> > Performing an svnadmin load results in the following error:
> >
> > ...
> >
> > <<< Started new transaction, based on original revision 650
> >
> > * editing path :
> > Dev/Common/Utils/External/StampVerData/MetaBuilderVersion.inf ... done.
> >
> > * editing path :
> > Dev/Common/Utils/External/StampVerData/MetaBuilderVersionDBG.inf ... done.
> >
> > * editing path : Dev/Common/Utils/External/StampVerData/OracleVersion.inf
> > ...svnadmin: E160000: SHA1 of reps '-1 0 45 132 a6ea37d29094deece56250ebe167ce46
> > 6a2d6bc4c7e58c0d2627a1b0dd5b23e542aec4d1 650-i2/_8' and '-1 136 45 132
> > a6ea37d29094deece56250ebe167ce46 6a2d6bc4c7e58c0d2627a1b0dd5b23e542aec4d1
> > 650-i2/_8' matches (6a2d6bc4c7e58c0d2627a1b0dd5b23e542aec4d1) but contents differ
> >
> > svnadmin: E160004: Filesystem is corrupt
> >
> > svnadmin: E200014: Checksum mismatch while reading representation:
> >
> > expected: a6ea37d29094deece56250ebe167ce46
> >
> > actual: 5f696f5d0755f3bcb5e927bdfba4bfa8
> >
> > In order to reproduce the error, you can use the attached VersionStamps3
> > repository dump file. It was taken with the same version of svnadmin from a
> > multi-GB repository that’s been around for over a decade and pruned
> > (svndumpfilter) to a small set of files and revisions in one folder while still
> > encountering the error. I first encountered the error another place very
> > quickly in an incremental load on top of a repo that was loaded from 1.8.14, but
> > that one was on a revision north of 120,000+. This is way easier to reproduce.
> > I tried filtering the dump further and dropping empty revisions, but the error
> > did not occur in those cases. The error also did not occur using 1.8.17,
> > 1.9.5., and 1.9.6. It also did not occur in 1.8.18 if I set enable-rep-sharing
> > = false before performing the load.
> >
> > Repro.bat script:
> >
> > @echo off
> >
> > :defineCommands
> >
> > rem You might need to adjust these lines to point to your
> >
> > rem compiled-from-source Subversion binaries, if using those:
> >
> > for %%i in (svn.exe) do set SVN="%%~$PATH:i"
> >
> > for %%i in (svnadmin.exe) do set SVNADMIN="%%~$PATH:i"
> >
> > :defineUrls
> >
> > rem Only supported access method: file:// <file:///>. If http:// or svn://, then
> >
> > rem you'll have to configure it yourself first.
> >
> > set URL=file:///%CD%/repos
> >
> > set URL=%URL:\=/%
> >
> > echo Base url for repo: %URL%
> >
> > :cleanAllDirsAndCreateRepo
> >
> > if exist repos rmdir /s /q repos
> >
> > if exist import-me rmdir /s /q import-me
> >
> > if exist wc rmdir /s /q wc
> >
> > %SVNADMIN% create repos
> >
> > pause
> >
> > :prepareGreekTree
> >
> > echo Making a Tree for import...
> >
> > mkdir import-me\Dev\Common\Utils\External
> >
> > echo Importing it...
> >
> > cd import-me
> >
> > %SVN% import -q -m "Initial import." %URL%
> >
> > cd ..
> >
> > rem This is where your reproduction recipe goes.
> >
> > svnadmin --version
> >
> > svnadmin load repos < VersionStamps3
> >
> > goto :eof
> >
> > Thanks,
> >
> > David
> >
Received on 2017-07-30 19:46:39 CEST