concurrent access produces "Filesystem has no such representation"

Hello

You may recall I have a perl script that I used some months ago to
test concurrent access to a repository. I have just tried this script
again using rev 1906 and I can provoke the following error

At revision 11.
Committing:
Sending wcstress.20178/bar1/foo1
Sending wcstress.20178/bar1/foo2
Sending wcstress.20178/bar2/foo1
Sending wcstress.20178/bar2/foo2
Transmitting file data ..
../svn/subversion/libsvn_client/commit.c:602
svn_error: #21061 : <Filesystem has no such representation>
  Commit failed (details follow):

../svn/subversion/libsvn_fs/reps-table.c:86
svn_error: #21061 : <Filesystem has no such representation>
  svn_fs__read_rep: no such representation `2x'

Subsequent attempt to checkout a new working copy fail with the same
error.

This error appears to be reasonably reproducible, it has occurred
three times out of four runs of the script, once at revision 11 and
twice at revision 28. The other run reached revision 60 before I
stopped it.

The script stress.pl is used as follows: start with two xterms open on
the same directory. This directory should not contain any file or
directory called 'repostress' as the script will delete it. It should
not contain a file called 'stop' either. In the first xterm run the
command

        ./stress.pl -c -s1

This will create the repostress repository, check out a revision 0
working copy, populate it, and commit it. When the message "Committed
revision 1." appears run the following command in the second xterm

       ./stress.pl -s1

This checks-out a second working copy. Each script then continues to
make modifications and commit the changes. Sometimes a conflict will
occur

../svn/subversion/libsvn_ra_local/commit_editor.c:112
svn_error: #21079 : <Transaction is out of date>
  out of date: `bar1/foo2' in txn `29'

but it will get resolved.

Eventually, assumming you trigger the problem, the scripts will start
to display the "no such representation" error. Touch a file called
'stop' to cause the scripts to terminate.

#!/usr/bin/perl -w

# This script constructs a repository, and populates it with
# files. Then it loops making changes to a subset of the files and
# committing the tree. When two instances are run in parallel
# sometimes the commit will fail with a merge conflict. This is
# expected, and is automatically resolved by updating.

# The files start off containing:
# A0
# 0
# A1
# 1
# A2
# .
# .
# A9
# 9

# Each script has an ID in the range 0-9, and when it modifies a file
# it modifes the line that starts with its ID. Thus scripts with
# different IDs will make changes that can be merged automatically.

# The main loop is then:
#
# step 1: modify a random selection of files
#
# step 2: optional sleep or wait for RETURN keypress
#
# step 3: update the working copy automatically merging out-of-date files
#
# step 4: try to commit, if not successful go to step 3 otherwise go to step 1

# To allow break-out of potentially infinite loops, the script will
# terminate if it detects the presence of a "stop file", the path to
# which is specified with the -S option (default ./stop). This allows
# the script to be stopped without any danger of interrupting an svn
# sub-process, which experiment shows can cause problems with the
# database locking.

use Getopt::Std;
use File::Find;
use File::Path;
use Cwd;

# Repository check/create
sub init_repo
  {
    my ( $repo, $create ) = @_;
    if ( $create )
      {
        rmtree([$repo]) if -e $repo;
        my $svnadmin_cmd = "svnadmin create $repo";
        system( $svnadmin_cmd) and die "$svnadmin_cmd: failed\n";
      }
    else
      {
        my $svnadmin_cmd = "svnadmin youngest $repo";
        my $revision = readpipe $svnadmin_cmd;
        die "$svnadmin_cmd: failed\n" if not $revision =~ m{^[0-9]};
      }
    $repo = getcwd . "/$repo" if not $repo =~ m[^/];
    return $repo;
  }

# Check-out working copy
sub check_out
  {
    my ( $url ) = @_;
    my $wc_dir = "wcstress.$$";
    mkdir "$wc_dir", 0755 or die "mkdir stress.$$: $!\n";
    my $svn_cmd = "svn co $url -d $wc_dir";
    system( $svn_cmd ) and die "$svn_cmd: failed\n";
    return $wc_dir;
  }

# Print status and update. The update is to do any required merges.
sub status_update
  {
    my ( $wc_dir, $wait_for_key ) = @_;
    my $svn_cmd = "svn st -u $wc_dir";
    print "Status:\n";
    system( $svn_cmd ) and die "$svn_cmd: failed\n";
    print "Press return to update/commit\n" if $wait_for_key;
    read STDIN, $wait_for_key, 1 if $wait_for_key;
    print "Updating:\n";
    $svn_cmd = "svn up $wc_dir";
    system( $svn_cmd ) and die "$svn_cmd: failed\n";
  }

# Print status, update and commit. The update is to do any required merges.
sub status_update_commit
  {
    my ( $wc_dir, $wait_for_key ) = @_;
    status_update $wc_dir, $wait_for_key;
    print "Committing:\n";
    my $now_time = localtime;
    my $svn_cmd = "svn ci $wc_dir -m '$now_time'";
    return system( $svn_cmd );
  }

# Get a list of all versioned files in the working copy
{
  my @get_list_of_files_helper_array;
  sub GetListOfFilesHelper
    {
      $File::Find::prune = 1 if $File::Find::name =~ m[/.svn];
      return if $File::Find::prune or -d;
      push @get_list_of_files_helper_array, $File::Find::name;
    }
  sub GetListOfFiles
    {
      my ( $wc_dir ) = @_;
      @get_list_of_files_helper_array = ();
      find( \&GetListOfFilesHelper, $wc_dir);
      return @get_list_of_files_helper_array;
    }
}

# Populate a working copy
sub populate
  {
    my ( $dir, $dir_width, $file_width, $depth ) = @_;
    return if not $depth--;

    for $nfile ( 1..$file_width )
      {
        my $filename = "$dir/foo$nfile";
        open( FOO, ">$filename" ) or die "open $filename: $!\n";

        for $line ( 0..9 )
          {
            print FOO "A$line\n$line\n" or die "write to $filename: $!\n";
          }
        close FOO or die "close $filename:: $!\n";

        my $svn_cmd = "svn add $filename";
        system( $svn_cmd ) and die "$svn_cmd: failed\n";
      }

    if ( $depth )
      {
        for $ndir ( 1..$dir_width )
          {
            my $dirname = "$dir/bar$ndir";
            mkdir "$dirname", 0755 or die "mkdir $dirname: $!\n";

            my $svn_cmd = "svn add $dirname";
            system( $svn_cmd ) and die "$svn_cmd: failed\n";

            populate( "$dirname", $dir_width, $file_width, $depth );
          }
      }
  }

# Modify a versioned file in the working copy
sub ModFile
  {
    my ( $filename, $mod_number, $id ) = @_;

    # Read file into memory replacing the line that starts with our ID
    open( FOO, "<$filename" ) or die "open $filename: $!\n";
    @lines = map { s[(^$id.*)][$1,$mod_number]; $_ } <FOO>;
    close FOO or die "close $filename: $!\n";

    # Write the memory back to the file
    open( FOO, ">$filename" ) or die "open $filename: $!\n";
    print FOO or die "print $filename: $!\n" foreach @lines;
    close FOO or die "close $filename: $!\n";
  }

sub ParseCommandLine
  {
    my %cmd_opts;

    # defaults
    $cmd_opts{'D'} = 2; # number of subdirs per dir
    $cmd_opts{'F'} = 2; # number of files per dir
    $cmd_opts{'N'} = 2; # depth
    $cmd_opts{'R'} = "repostress"; # repository name
    $cmd_opts{'S'} = "stop"; # path of file to stop the script
    $cmd_opts{'U'} = "none"; # URL
    $cmd_opts{'c'} = 0; # create repository
    $cmd_opts{'i'} = 0; # ID
    $cmd_opts{'s'} = -1; # sleep interval
    $cmd_opts{'n'} = 200; # sets of changes
    $cmd_opts{'x'} = 4; # files to modify

    getopts( 'ci:n:s:x:D:F:N:R:U:', \%cmd_opts )
      or die "
usage: stress [-c] [-i num] [-n num] [-s secs] [-x num]
              [-D num] [-F num] [-N num] [-R path] [-S path] [-U url]
where
  -c cause repository creation
  -i the ID (default is 5 if -c given 1 otherwise)
  -n the number of sets of changes to commit
  -s the sleep delay (-1 wait for key, 0 none)
  -x the number of files to modify
  -D the number of sub-directories per directory in the tree
  -F the number of files per directory in the tree
  -N the depth of the tree
  -R the path to the repository
  -S the path to the file whose presence stops this script
  -U the URL to the repository (file:///<-R path> by default)
";

    # default ID if not set
    $cmd_opts{'i'} = 1 + 5 * $cmd_opts{'c'} if not $cmd_opts{'i'};

    return %cmd_opts;
  }

############################################################################
# Main

srand 123456789;

my %cmd_opts = ParseCommandLine();

my $repo = init_repo $cmd_opts{'R'}, $cmd_opts{'c'};

# Make URL from path if URL not explicitly specified
$cmd_opts{'U'} = "file://$repo" if $cmd_opts{'U'} eq "none";

my $wc_dir = check_out $cmd_opts{'U'};

if ( $cmd_opts{'c'} )
  {
    populate $wc_dir, $cmd_opts{'D'}, $cmd_opts{'F'}, $cmd_opts{'N'};
    status_update_commit $wc_dir, 0 and die "populate checkin failed\n";
  }

my @wc_files = GetListOfFiles $wc_dir;
die "not enough files in repository\n" if $#wc_files < $cmd_opts{'x'};

my $wait_for_key = $cmd_opts{'s'} < 0;

my $stop_file = $cmd_opts{'S'};

for $mod_number ( 1..$cmd_opts{'n'} )
  {
    my @chosen;
    for ( 1..$cmd_opts{'x'} )
      {
        # Extract random file from list and modify it
        my $mod_file = splice @wc_files, int rand $#wc_files, 1;
        ModFile $mod_file, $mod_number, $cmd_opts{'i'};
        push @chosen, $mod_file;
      }
    # Reinstate list of files, the order doesn't matter
    push @wc_files, @chosen;

    if ( $cmd_opts{'x'} > 0 ) {
      # Loop committing until successful or the stop file is created
      1 while not -e $stop_file and status_update_commit $wc_dir, $wait_for_key;
    } else {
      status_update $wc_dir, $wait_for_key;
    }
    sleep $cmd_opts{'s'} if $cmd_opts{'s'} > 0;

    last if -e $stop_file;
  }

-- 
Philip
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org