[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Issue creating large # of nodes in a transaction. (SWIG perl bindings)

From: Dan Mercer <dmercer_at_8kb.net>
Date: 2006-03-11 00:08:56 CET

Hi folks,

I have a tool that imports files into a subversion repository. Each
import can consist of a large number of files and they should be
imported in the same transaction. Unfortunately, my code is consistently
crashing on any transaction that attempts to create > ~1400 new nodes
(files or directories). Here are the particulars:

* Subversion version 1.3.0 (r17949)
* FSFS Repository
* Swig perl bindings (Perl 5.8.5)
* FreeBSD 4.11

And here is a very simple recipe to reproduce what I am seeing. Run on a
newly created fsfs repository as '$script /path/to/repos $count':

#!/usr/local/bin/perl -w

use strict;

use SVN::Core;
use SVN::Fs;
use SVN::Repos;

$|=1;

my $repos_path = shift;
my $count = shift;

my $svn = SVN::Repos::open($repos_path);
my $youngest = $svn->fs->youngest_rev();
my $rev_root = $svn->fs->revision_root($youngest);
my $txn = $svn->fs_begin_txn_for_commit($youngest, 'nobody', '');
my $txn_root = SVN::Fs::txn_root($txn);

my $nc = 0;
while ($nc < $count) {
    my $path = "/".$nc."_node";
    my $path_type = SVN::Fs::check_path($rev_root, $path);
    if ($path_type == $SVN::Node::none) {
# SVN::Fs::make_dir($txn_root, $path);
        SVN::Fs::make_file($txn_root, $path);
        print "$nc,",
    }
    $nc++;
}
my $revnum = $svn->fs_commit_txn($txn);
print "New revision -> $revnum\n";

--
output:
$ svnadmin create --fs-type fsfs /tmp/test1
$ ./test.pl /tmp/test2 2048
0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,
...SNIP...
1406,1407,1408,1409,1410,1411,Abort trap (core dumped)
$
$ ./test.pl /tmp/test
I'm not sure what I'm doing wrong here, but any pointers would be 
helpful, even if you can just verify that this code behaves the same in 
your environment. It could be that my assumption that a single 
transaction should be able to handle many (certainly more than a few 
thousands) events is incorrect as well.
 
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Sat Mar 11 00:08:21 2006

This is an archived mail posted to the Subversion Users mailing list.