[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Fwd: Error during repetative calls to cat

From: Ed Hillmann <edhillmann_at_yahoo.com>
Date: 2005-06-17 07:44:35 CEST

More fun. I've looked at the generated SWIG files
used to bind the Perl environment to the svn libs.
The generated function for cat looks like...

XS(_wrap_svn_client_cat) {
    {
        svn_stream_t *arg1 = (svn_stream_t *) 0 ;
        char *arg2 = (char *) 0 ;
        svn_opt_revision_t *arg3 = (svn_opt_revision_t
*) 0 ;
        svn_client_ctx_t *arg4 = (svn_client_ctx_t *)
0 ;
        apr_pool_t *arg5 = (apr_pool_t *) 0 ;
        svn_error_t *result;
        apr_pool_t *_global_pool ;
        svn_opt_revision_t rev3 ;
        int argvi = 0;
        dXSARGS;
        
        {
            _global_pool = arg5 =
svn_swig_pl_make_pool (ST(items-1));
        }
        if ((items < 4) || (items > 5)) {
            SWIG_croak("Usage:
svn_client_cat(out,path_or_url,revision,ctx,pool);");
        }
        {
            svn_swig_pl_make_stream (&arg1, ST(0));
        }
        if (!SvOK((SV*) ST(1))) arg2 = 0;
        else arg2 = (char *) SvPV(ST(1), PL_na);
        {
            arg3 = &rev3;
            if (ST(2) == NULL || ST(2) == &PL_sv_undef
|| !SvOK(ST(2))) {
                rev3.kind =
svn_opt_revision_unspecified;
            }
            else if (sv_isobject(ST(2)) &&
sv_derived_from(ST(2), "_p_svn_opt_revision_t")) {
                SWIG_ConvertPtr(ST(2), (void **)&arg3,
SWIGTYPE_p_svn_opt_revision_t, 0);
            }
            else if (looks_like_number(ST(2))) {
                rev3.kind = svn_opt_revision_number;
                rev3.value.number = SvIV(ST(2));
            }
            else if (SvPOK(ST(2))) {
                char *input = SvPV_nolen(ST(2));
                if (strcasecmp(input, "BASE") == 0)
                rev3.kind = svn_opt_revision_base;
                else if (strcasecmp(input, "HEAD") ==
0)
                rev3.kind = svn_opt_revision_head;
                else if (strcasecmp(input, "WORKING")
== 0)
                rev3.kind = svn_opt_revision_working;
                else if (strcasecmp(input,
"COMMITTED") == 0)
                rev3.kind =
svn_opt_revision_committed;
                else if (strcasecmp(input, "PREV") ==
0)
                rev3.kind = svn_opt_revision_previous;
                else if (*input == '{') {
                    svn_boolean_t matched;
                    apr_time_t tm;
                    svn_error_t *err;
                    
                    char *end = strchr(input,'}');
                    if (!end)
                    SWIG_croak("unknown opt_revision_t
type");
                    *end = '\0';
                    err = svn_parse_date (&matched,
&tm, input + 1, apr_time_now(),
                    svn_swig_pl_make_pool ((SV
*)NULL));
                    if (err) {
                        svn_error_clear (err);
                        SWIG_croak("unknown
opt_revision_t type");
                    }
                    if (!matched)
                    SWIG_croak("unknown opt_revision_t
type");
                    
                    rev3.kind = svn_opt_revision_date;
                    rev3.value.date = tm;
                } else
                SWIG_croak("unknown opt_revison_t
type");
            } else
            SWIG_croak("unknown opt_revision_t type");
        }
        {
            if (SWIG_ConvertPtr(ST(3), (void **)
&arg4, SWIGTYPE_p_svn_client_ctx_t,0) < 0) {
                SWIG_croak("Type error in argument 4
of svn_client_cat. Expected _p_svn_client_ctx_t");
            }
        }
        if (items > 4) {
            
        }
        {
            result = (svn_error_t
*)svn_client_cat(arg1,(char const
*)arg2,(svn_opt_revision_t const *)arg3,arg4,arg5);
            
            
            
            
        }
        {
            if (result) {
                SV *exception_handler = perl_get_sv
("SVN::Error::handler", FALSE);
                
                if (SvOK(exception_handler)) {
                    SV *callback_result;
                    
                    svn_swig_pl_callback_thunk
(CALL_SV, exception_handler,
                    &callback_result, "S", result,
                    SWIGTYPE_p_svn_error_t);
                } else {
                    ST(argvi) = sv_newmortal();
                    SWIG_MakePtr (ST(argvi), (void
*)result, SWIGTYPE_p_svn_error_t ,0);
                    argvi++;
                }
            }
        }
        XSRETURN(argvi);
        fail:
        ;
    }
    croak(Nullch);
}

One thing I noticed, was that it looks like it's
calling

svn_swig_pl_make_stream (&arg1, ST(0));

everytime, but never closing the stream. I think.
I've never read this SWIG code before, so I could be
wrong.

Shouldn't this clean up after itself a little better?
Or is it ok, and I'm just not seeing it?

Thanks,
Ed

Send instant messages to your online friends http://au.messenger.yahoo.com

attached mail follows:


I've found some other messages where people were
getting the same kind of error. And it looks like it
has to do with the system limit on simultaneously open
files.

When I run the script

#!/usr/bin/perl -w
use strict;
use IO::File;
my @tempfiles;
for my $i (1 .. 2000) {
my $fh = IO::File->new_tmpfile();
die "Failed to created file number $i.\n" unless
defined $fh;
push @tempfiles, $fh;
}

I get

Failed to created file number 254.

So, there is a system limit, and I don't think I can
change that.

However, my test of ctx->cat is simply opening the
same file repeatedly. Does that mean that the cat
command in the libsvn_client library (which is being
linked to by the perl script) is not cleaning up any
handles?

I know I'm closing the handle that I open. Would
there be something else in the local File support that
is opening something and not closing?

I guess I should try doing the same thing in C, but if
anyone has any clues, I'd love any insight.

Thanks,
Ed

Send instant messages to your online friends http://au.messenger.yahoo.com

attached mail follows:


I'm using the Perl bindings to access a SVN
repository. I'm doing performance timings, so I have
a simple loop that goes through for a single file, and
isses the same command multiple times....

sub test_svn_cat {
    my $srcUrl = shift;
    my $srcRevision = shift;
    my $targetPath = shift;
    
    my $err;

    open(OUTPUT_HANDLE, ">$targetPath/svnOutput.txt");
    $err = $ctx->cat(\*OUTPUT_HANDLE, $srcUrl,
$srcRevision);

    if (!close(OUTPUT_HANDLE)) {
        print "Close failed: $!\n";
        return 0;
    }
    return 1;
}

sub test_svn_cat_loop {
    my $srcUrl = shift;
    my $srcRevision = shift;
    my $targetPath = shift;
    my $svnCatFn = \&test_svn_cat;

    print "\nTesting SVN Cat ($ITERATIONS
iterations)\n";
    print "Getting Revision $srcRevision of $srcUrl
...\n";

        
    my $t0 = [gettimeofday];

    my $local_iter = 0;
    while ($local_iter < $ITERATIONS) {
            &$svnCatFn($srcUrl, $srcRevision, $targetPath);
        $local_iter++;
    }
    
    my $t1 = [gettimeofday];
    
    $t0_t1 = tv_interval $t0, $t1;
    
    my $avg = $t0_t1 / $ITERATIONS;
    
    print "Test SVN Cat\n";
    print "------------\n";
    print " # of iters: $ITERATIONS\n";
    print " elapsed time (seconds): $t0_t1\n";
    print " avg: $avg\n";
    
}
test_svn_cat_loop
('file:///ct/acct/ehillman/svnRepos/trunk/conftool/docobj/build.xml',
16009,
'/ct/acct/ehillman/svnPerformance/perl/output');

When the URL being passed into the cat method is an
svn url (for example,
"svn://machineName/trunk/conftool/docobj/build.xml"),
it works fine. My default iteration count is 100.

When I pass it a file URL (for example,
"file:///ct/acct/ehillman/svnRepos/trunk/conftool/docobj/build.xml"),
it fails on the 43rd iteration. So, if my iteration
count is less than that, it works without a problem.
But once it hits that number, I get an error.

It's also complaining when it fails that it can't find
Carp/Heavy.pm, which I don't understand because it's
in its PERL5LIB.

Anyways, has anyone come across this before?

Thanks,
Ed

Send instant messages to your online friends http://au.messenger.yahoo.com

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Fri Jun 17 07:46:20 2005

This is an archived mail posted to the Subversion Users mailing list.