RE: Trival merge of big text file: Dismal performance, 540x faster if binary.

Hello, Stefan,

> Can you share both versions of this file ... ?

The bad news: No, sorry.

The good news: This is a generic problem. So it is quite easy to
come up with example files.

To be more specific than that, the following Perl script
generates two files (which aren't XML, but just lines with
numbers) that trigger the same SVN behavior. Binary merge: 3.6
seconds. Text merge: Longer than it takes to clean up the Perl
script, running it again to check it still works, and typing this
text to accompany it, finding I goofed, and sending the corrected
message again. :-).

Regards, Andreas


#!/usr/bin/perl -w

# Generate stupid files on stdout.

use strict;

# For the overhauled file, set to 1:
my $overhaul = 0;

my $number = 1;

for (1 .. 1000000) {

    # 1073741824 and 910111213 have no common divisor,
    # so this will take a while before it repeats.
    $number = ($number + 910111213) % 1073741824;

    my $printme;
    if($overhaul) {
        $printme = ($number % 4 != 0 ? $number * 13 % 1073741824 : $number);
    } else {
        $printme = $number;
    }
    print $printme,"\n";
}

--
Dr. Andreas Krüger, Senior Developer

Tel. (+49) (211) 280 69-1132
andreas.krueger_at_hp.com

DV-RATIO NORDWEST GmbH, Habsburgerstraße 12, 40547 Düsseldorf, Germany
 
für
 
Hewlett-Packard GmbH H Herrenberger Str. 140 71034 Böblingen www.hp.com/de
Geschäftsführer: Volker Smid (Vorsitzender), Michael Eberhardt, Thorsten Herrmann,
Martin Kinne, Heiko Meyer, Ernst Reichart, Rainer Sterk
Vorsitzender des Aufsichtsrates: Jörg Menno Harms
Sitz der Gesellschaft: Böblingen S Amtsgericht Stuttgart HRB 244081 WEEE-Reg.-Nr. DE 30409072



-----Original Message-----
From: krueger, Andreas (Andreas Krüger, DV-RATIO)
Sent: Thursday, January 13, 2011 4:03 PM
To: 'users_at_subversion.apache.org'
Subject: RE: Trival merge of big text file: Dismal performance, 540x faster if binary.

Hello, Stefan,

> Can you share both versions of this file ... ?

The bad news: No, sorry.

The good news: This is a generic problem. So it is quite easy to come
up with example files.

To be more specific than that, the following Perl script generates two
files (which aren't XML, but just lines with numbers) that trigger the
same SVN behavior. Binary merge: 3.6 seconds. Text merge: Longer
than it takes to clean up the Perl script, running it again to check
it still works, and typing this text to accompany it :-) .

Regards, Andreas


#!/usr/bin/perl -w

# Generate stupid files on stdout.

use strict;

# For the overhauled file, set to 1:
my $overhaul = 0;

my $number = 1;

for (1 .. 1000000) {

    # 1073741824 and 910111213 have no common divisor,
    # so this will take a while before it repeats.
    $number = ($number + 910111213) % 1073741824;

--
Dr. Andreas Krüger, Senior Developer

Tel. (+49) (211) 280 69-1132
andreas.krueger_at_hp.com

DV-RATIO NORDWEST GmbH, Habsburgerstraße 12, 40547 Düsseldorf, Germany
 
für
 
Hewlett-Packard GmbH H Herrenberger Str. 140 71034 Böblingen www.hp.com/de
Geschäftsführer: Volker Smid (Vorsitzender), Michael Eberhardt, Thorsten Herrmann,
Martin Kinne, Heiko Meyer, Ernst Reichart, Rainer Sterk
Vorsitzender des Aufsichtsrates: Jörg Menno Harms
Sitz der Gesellschaft: Böblingen S Amtsgericht Stuttgart HRB 244081 WEEE-Reg.-Nr. DE 30409072
Received on 2011-01-13 16:09:19 CET