Hello,
We've been running Subversion 1.0.4 for about a month now, with all access
via webdav on Apache/2.0.48. Just recently, we started receiving errors. Any
operation that contacts the repository yields an error. From the client end,
the error looks like this:
$ svn status -u
svn: PROPFIND request failed on '/sync/sync/trunk/code'
svn:
Could not open the requested SVN filesystem
From Apache's logs, the error looks like this:
[Thu Jul 29 10:09:48 2004] [error] [client 10.20.87.90] (20014)Error string
not specified yet: Berkeley DB error while opening environment for
filesystem /local/svn/sync/db:\nDB_RUNRECOVERY: Fatal error, run database
recovery
[Thu Jul 29 10:09:48 2004] [error] [client 10.20.87.90] Could not fetch
resource information. [500, #0]
[Thu Jul 29 10:09:48 2004] [error] [client 10.20.87.90] Could not open the
requested SVN filesystem [500, #160029]
[Thu Jul 29 10:09:48 2004] [error] [client 10.20.87.90] (17)File exists:
Could not open the requested SVN filesystem [500, #160029]
Once we get the first error, it's all errors. Running "svnadmin recover"
does not fix the problem.
I tried restoring the repository from a hot backup (created using
hot-backup.py). After that, I could get a fresh checkout and then run this
for 10-20 seconds (approx. 10-20 iterations):
$ while true; do svn status -u; done;
Then I would get the same error, and it was all errors from there. This
happened whether I restored to a backup 3 days old or 3 weeks old. It didn't
make any difference whether or not I applied incremental backups on top of
that (using svnadmin load). So all this suggests to me that the problem is
not a "bad checkin" or other recent database corruption. Otheriwse, why
would a snapshot that worked for 3 weeks suddenly fail in a few seconds?
I've tried this from several different client machines, and I get the same
results in all cases.
I did notice that occasionally, the last *successful* client operation would
print trouble in the apache error log. This was often different, but always
related to closing one berkeley db database or another. For example:
[Thu Jul 29 09:29:24 2004] [error] [client 10.20.87.90] (20014)Error string
not specified yet: Berkeley DB error while closing 'nodes' database for
filesystem /local/svn/sync/db:\nDB_RUNRECOVERY: Fatal error, run database
recovery
This, I think, is the root of the problem. Here is another version:
[Thu Jul 29 10:09:43 2004] [error] [client 10.20.87.90] (20014)Error string
not specified yet: Berkeley DB error while closing 'uuids' database for
filesystem /local/svn/sync/db:\nDB_RUNRECOVERY: Fatal error, run database
recovery
[Thu Jul 29 10:09:43 2004] [error] [client 10.20.87.90] (20014)Error string
not specified yet: Berkeley DB error while closing 'representations'
database for filesystem /local/svn/sync/db:\nDB_RUNRECOVERY: Fatal error,
run database recovery
[Thu Jul 29 10:09:43 2004] [error] [client 10.20.87.90] (20014)Error string
not specified yet: Berkeley DB error while closing 'strings' database for
filesystem /local/svn/sync/db:\nDB_RUNRECOVERY: Fatal error, run database
recovery
[Thu Jul 29 10:09:43 2004] [error] [client 10.20.87.90] (20014)Error string
not specified yet: Berkeley DB error while closing 'representations'
database for filesystem /local/svn/sync/db:\nDB_RUNRECOVERY: Fatal error,
run database recovery
[Thu Jul 29 10:09:43 2004] [error] [client 10.20.87.90] (20014)Error string
not specified yet: Berkeley DB error while closing 'nodes' database for
filesystem /local/svn/sync/db:\nDB_RUNRECOVERY: Fatal error, run database
recovery
I don't know why this time we got five errors instead of only one. Another
time I got two lines both with the "nodes" database:
[Thu Jul 29 09:50:29 2004] [error] [client 208.204.52.91] (20014)Error
string not specified yet: Berkeley DB error while closing 'nodes' database
for filesystem /local/svn/sync/db:\nDB_RUNRECOVERY: Fatal error, run
database recovery
[Thu Jul 29 09:50:29 2004] [error] [client 208.204.52.91] (20014)Error
string not specified yet: Berkeley DB error while closing 'nodes' database
for filesystem /local/svn/sync/db:\nDB_RUNRECOVERY: Fatal error, run
database recovery
I suspect that whatever goes wrong here is the cause of the subsequent
problems.
Next I tried switching from webdav to svnserve. If I restored my repository
from a backup and then accessed it via only svnserve, there were no
problems. My little loop ran for 7 minutes before I Ctrl-C'd it. Then I
launched apache again and saw the errors in the usual amount of time. Once
those errors appeared, accessing the repository via svnserve would fail,
too. So apparently the webdav access is corrupting the repository.
Nothing on the machine changed to cause this sudden failure of webdav.
Really. :-)
Here is the relevant snippet from our apache config file:
<Location /sync>
DAV svn
SVNPath /local/svn/sync
SVNAutoversioning on
AuthName "Subversion repositories"
AuthType Basic
AuthUserFile "/local/svn/.syncpasswd"
<LimitExcept GET PROPFIND OPTIONS REPORT>
Require valid-user
</LimitExcept>
</Location>
I've run ldd against svn, svnserve, mod_dav_svn.so, and mod_authz_svn.so,
and they are all linked as follows:
libdb-4.2.so => /lib/tls/libdb-4.2.so (0xb7375000)
We are not storing our berkeley db files on a network share, but on disk
local to the server.
So, any suggestions?
Thanks,
Paul Jungwirth
_________________________________________________________________
FREE pop-up blocking with the new MSN Toolbar – get it now!
http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Thu Jul 29 17:57:43 2004