--On samedi 19 février 2005 10:11 -0500 Mark Benedetto King
<mbk@lowlatency.com> wrote:
> On Sat, Feb 19, 2005 at 10:32:49AM +0100, Michel Jouvin wrote:
>> >
>> > The first corruptions we experienced generally occurred during commit,
>> > especially on large repositories. When we looked at possible causes for
>> > these corruptions, we found that one reason was we were running 2
>> > Apache servers on 2 different nodes in a cluster configuration
>> > (cluster file system, no NFS involved). We shut down one of the server
>> > and it more or less solved the corruption during commits. This remains
>> > strange as the cluster file system has a pure local file system
>> > semantics and we never experienced such problems with other databases
>> > or other Db usage.
>> >
>
> Which cluster filesystem?
Tru64/TruCluster Cluster File System. I am not sure about Florian question
: the file system (including locks) has the whole semantic of a local file
system but that I am not sure about system mutexes. In fact it is possible
to create shared memory between cluster nodes but it has to be done
explicitly.
Anyway, as I explained below, we now run a configuration where we only have
one Apache server handling subversion request and I have httpd logs to
prove that. I understand that there is may be something wrong in our
configuration if we run several concurrent Apachae instances on several
nodes accessing the same repositories. But the database corruption we have
now are not in this configuration and even not related to commit or other
repository modification.
>
>> > Now we experienced corruptions not related to any repository write. We
>> > have log files showing successful repository access through HTTP GET
>> > followed by a GET failure due to database corruption without any
>> > repository modification in between and without any Apache
>> > problem/restart. We suspected that these corruption were related to
>> > Apache restart during a transaction but we now have evidence that
>> > corruption can occur at any time without any repository modification.
>> > We have Apache log files and corrupted repository copies.
>
> Is this with two httpd nodes or one?
>
> Is the httpd being used for anything else (CGI, PHP, etc?), or is it
> dedicated for mod_dav_svn use?
>
Subversion server is implemented through a virtual host that is handled by
only one node running only one Apache instance (with threads worker). This
server also handles requests for a lot of other Web apps, including CGI,
PHP.
>> >
>> > Generally svnadmin recover fails on these corruptions. Sometimes we
>> > were able to fix corruptions by recover + verify as documented in a
>> > note. We also have a directory that we restored from backup and needed
>> > to repair before having it accessible again. In this case we had to
>> > use recover + verify. And verify + recover definitly corrupts the
>> > repository.
>
> What error message do you get when recover fails?
>
Attached are 2 recover outputs with the same database. The first one
(*-success) was doing recover (which fails) followed by verify (which
succeeds) followed by another recover (with succeed without doing any
recovery).
The second log file (*-error) corresponds to a verify done first. In this
case the database is impossible to recover.
I have a backup of the database before doing recovery, I can send it if
this is useful.
>
> --ben
>
Michel
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
*************************************************************
* Michel Jouvin Email : jouvin@lal.in2p3.fr *
* LAL / CNRS Tel : +33 1 64468932 *
* B.P. 34 Fax : +33 1 69079404 *
* 91898 Orsay Cedex *
* France *
*************************************************************
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sat Feb 19 20:08:34 2005