Hi,
> On Tue, Aug 28, 2001 at 09:44:43PM -0700, Greg Stein wrote:
>>...
>> Given that, I began running the mass-commit script again. We
>> still have the two remaining problems:
>>
>> * WC locks bugging up the commits
>> * the DB_INCOMPLETE thang
>>
>> The latter is actually posing a significant problem now. I kept
>> running out
>>...
>> Oops! What I found instead was a number of file descriptors
>> pointing to the DB files. These leaked file descriptors also
>> referenced the log files which I was trying to delete... Of
>> course, the system still referred to them, so they were still
>> using disk space.
>>...
>> I am now suspecting the leaked descriptors are because we bail
>> out during the DB->close routines. When one of them returns
>> DB_INCOMPLETE, I think the logic just bails out (gotta go look).
>>
>> Regardless of what is happening, we have to get those descriptors
>> closed. We can't be leaking descriptors on the server like that.
>> So... that is my current task.
>
> I just added a "sleep 1" after each commit in the mass-commit script. This
> prevented all occurances of the DB_INCOMPLETE error. Further, it prevented
> the descriptor leak.
>
> [ basically, it gave a pause to let Apache clean up the previous request
> before processing the next commit request ]
>
> Thus: simultaneous access is causing DB_INCOMPLETE (not a surprise, given
> the Berkeley docs), the DB_INCOMPLETE is messing up our close, and we are
> leaking.
Yes, that is what is happening. The svn_fs_close_fs ends up calling
cleanup_fs_apr (via the apr_pool_destroy of the fs pool). This in turn
calls cleanup_fs. This call has a series of SVN_ERR wrapped db->close
calls. Ofcourse when another thread is writing to the db DB_INCOMPLETE
is returned and cleanup_fs bails out.
The docs state DB_INCOMPLETE can be safely ignored in db->close and
db->sync. These urls might be helpfull (and important regarding corruption/
recovery):
http://www.sleepycat.com/docs/ref/program/errorret.html#DB_INCOMPLETE
http://www.sleepycat.com/docs/api_c/db_close.html
With this in mind there are 2 solutions for the problem. The first is
to tell the db not to flush (so no DB_INCOMPLETE can occur). The second
is _try_ to flush but ignore DB_INCOMPLETE when flushing doesn't work.
Solution 1:
--- libsvn_fs/fs.c Tue Aug 28 13:08:41 2001
+++ libsvn_fs/fs.c~ Wed Aug 29 12:40:24 2001
@@ -91,7 +91,7 @@
char *msg = apr_psprintf (fs->pool, "closing `%s' database", name);
*db_ptr = 0;
- SVN_ERR (DB_WRAP (fs, msg, db->close (db, 0)));
+ SVN_ERR (DB_WRAP (fs, msg, db->close (db, DB_NOSYNC)));
}
return SVN_NO_ERROR;
Solution 2:
--- libsvn_fs/fs.c Tue Aug 28 13:08:41 2001
+++ libsvn_fs/fs.c~ Wed Aug 29 12:28:13 2001
@@ -89,9 +89,18 @@
{
DB *db = *db_ptr;
char *msg = apr_psprintf (fs->pool, "closing `%s' database", name);
+ int db_err;
*db_ptr = 0;
- SVN_ERR (DB_WRAP (fs, msg, db->close (db, 0)));
+ db_err = db->close(db, 0);
+
+ /* According to the Berkeley documentation it is safe to
+ ignore DB_INCOMPLETE on db->close and db->sync.
+ */
+ if (db_err == DB_INCOMPLETE)
+ db_err = 0;
+
+ SVN_ERR (DB_WRAP (fs, msg, db_err)));
}
return SVN_NO_ERROR;
> While I'm going to look into this tomorrow, I'll note that it is
> not really
> a showstopper for M3. The database is not damaged, content is fine, and
> further processing is fine. While we may leak descriptors and hold open
> logs, it is very easy to have Apache recycle the processes to clean these
> up. I *do* expect to get the close mostly fixed up, but I also want to
> ensure that people have it in the proper perspective.
>
>
> More and varied testing is, of course, still called for before we shift to
> self-hosting. At this point, we do not have any known showstoppers. That
> means we have 5 left ;-)
>
> Cheers,
> -g
Keep it up, M3 seems really close now :)
Sander
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sat Oct 21 14:36:37 2006