AW: svn client fails with Berkeley DB error Cannot allocate memor y - repository wedges

Hi,

If you call your script recursivly,
it could end up having _many_ "open connections"....

Locks cannot be released until the scripts ends,
and the script only ends when a subprocess is done.
I can imagine that 4000 is a limmit way to small on
a medium sized repository.

The way you describe your sript it should release
the subversion server after each "svn ls",
but maybe something is wrong here.
Exausting your resources is very simple using recursion :-)

Maybe you should rewrite your script to first create
an infomation file about which parts of the tree to access.
Then rocess this info afterwards,
thus separating information gathering, and information processing.

Huibert Alblas

-- 
To undestand recursion,
we must first understand recursion
 
-----Ursprüngliche Nachricht-----
Von: Martin J. Evans [mailto:martin.evans@easysoft.com] 
Gesendet: Dienstag, 1. Juni 2004 10:10
An: users@subversion.tigris.org
Cc: sussman@collab.net
Betreff: Re: svn client fails with Berkeley DB error Cannot allocate memory
- repository wedges
Ben,
On 31-May-2004 Ben Collins-Sussman wrote:
> On Fri, 2004-05-28 at 06:06, Martin J. Evans wrote:
>>  Berkeley DB error while opening 'transactions' table for filesys tem 
>> /var/subversion/distribution/linux-x86/db:
>> Cannot allocate memory
> 
> IIRC, this error often means that BDB has run out of lock objects.  
> Try increasing the the number of locks in your repos/db/DB_CONFIG file 
> and then run 'svnadmin recover'.
Thanks for that. I take you are referring to:
set_lk_max_locks   2000
set_lk_max_lockers 2000
set_lk_max_objects 2000
Bumping all of these to 4000 did not help. When using file URL:
Couldn't open a repository: Unable to open an ra_local session to URL:
Unable to open repository
'file:///var/subversion/distribution/linux-x86/common/libraries/extras/tags'
:
Berkeley DB error while opening 'uuids' table for filesystem
/var/subversion/distribution/linux-x86/db:
Cannot allocate memory at /tmp/x.pl line 11
line 11 is an SVN::Client ls call.
Once I've run the script:
sh-2.05$ svnadmin recover linux-x86 
Please wait; recovering the repository may take some time...
svn: DB_RUNRECOVERY: Fatal error, run database recovery
The last lines of strace show (I don't see the reason for the carp):
stat64("/var/subversion/distribution/linux-x86/db/nodes",
{st_mode=S_IFREG|0644, st_size=86016, ...}) = 0
stat64("/var/subversion/distribution/linux-x86/db/nodes",
{st_mode=S_IFREG|0644, st_size=86016, ...}) = 0
open("/var/subversion/distribution/linux-x86/db/nodes", O_RDWR|O_LARGEFILE)
= 4
fcntl64(4, F_SETFD, FD_CLOEXEC)         = 0
read(4, "\6\0\0\0$\364\0\0\0\0\0\0b1\5\0\t\0\0\0\0\20\0\0\0\t\0"..., 512) =
512
close(4)                                = 0
stat64("/var/subversion/distribution/linux-x86/db/nodes",
{st_mode=S_IFREG|0644, st_size=86016, ...}) = 0
open("/var/subversion/distribution/linux-x86/db/nodes", O_RDWR|O_LARGEFILE)
= 4
fcntl64(4, F_SETFD, FD_CLOEXEC)         = 0
fstat64(4, {st_mode=S_IFREG|0644, st_size=86016, ...}) = 0
brk(0x8354000)                          = 0x8354000
munmap(0x40640000, 16384)               = 0
munmap(0x4048a000, 327680)              = 0
munmap(0x404da000, 1466368)             = 0
close(4)                                = 0
munmap(0x40448000, 270336)              = 0
munmap(0x40018000, 8192)                = 0
stat64("/usr/local/lib/perl5/5.8.2/i686-linux/Carp/Heavy.pmc", 0xbfffef5c) =
-1 ENOENT (No such file or directory)
open("/usr/local/lib/perl5/5.8.2/i686-linux/Carp/Heavy.pm",
O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
stat64("/usr/local/lib/perl5/5.8.2/Carp/Heavy.pmc", 0xbfffef5c) = -1 ENOENT
(No such file or directory) open("/usr/local/lib/perl5/5.8.2/Carp/Heavy.pm",
O_RDONLY|O_LARGEFILE) = 4
brk(0x8355000)                          = 0x8355000
fstat64(4, {st_mode=S_IFREG|0444, st_size=5918, ...}) = 0 old_mmap(NULL,
4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40018000
read(4, "# Carp::Heavy uses some variable"..., 4096) = 4096
brk(0x8357000)                          = 0x8357000
brk(0x8358000)                          = 0x8358000
brk(0x8359000)                          = 0x8359000
brk(0x835a000)                          = 0x835a000
brk(0x835b000)                          = 0x835b000
brk(0x835c000)                          = 0x835c000
brk(0x835d000)                          = 0x835d000
brk(0x835e000)                          = 0x835e000
brk(0x835f000)                          = 0x835f000
brk(0x8360000)                          = 0x8360000
brk(0x8361000)                          = 0x8361000
brk(0x8362000)                          = 0x8362000
read(4, ";\n    my $caller = caller($i);\n "..., 4096) = 1822
brk(0x8363000)                          = 0x8363000
brk(0x8364000)                          = 0x8364000
brk(0x8365000)                          = 0x8365000
brk(0x8366000)                          = 0x8366000
read(4, "", 4096)                       = 0
close(4)                                = 0
munmap(0x40018000, 4096)                = 0
write(2, "Couldn\'t open a repository: Unab"..., 294) = 294 fcntl64(3,
F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}) = 0
close(3)                                = 0
munmap(0x40017000, 4096)                = 0
_exit(255)                              = ?
> I'd like to know what your script is doing (or what Web::SVN is doing) 
> that is so lock-intensive...
The script simply does an ls on the repository then walks down the
files/dirs recursively calling itself on dirs. However, for each file/dir it
gets svn:externals to see if any are defined. The idea was to produce a tree
of the repository because the repository has a large number of svn:externals
references. I can mail you seperately with the tree output which would give
you an idea of the repository structure. The script was in my last email.
BTW, all that is required is running the script - there is no one else
accessing the repository.
Let me know if these is any further info that could help.
Martin
--
Martin J. Evans
Easysoft Ltd, UK
Development
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org