[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

sloooowww import

From: Mark Richards <mark.richards_at_massmicro.com>
Date: 2006-06-01 00:31:24 CEST

I am perhaps guilty of providing too much information. But I know that
what I am including below is likely to be needed eventually, so at the
risk of being flamed, I'm including it.

The issue I'm having trouble with is very slow repository import
performance. I'll describe the setup, show the details, and ask if
anyone can shed some light. I've done my due diligence, checking the
bugs list and searching archives. Nothing stands out.

I've already done one section of my source tree (which took a few hours
and finally completed without error). This exhibited the same issues
I'm reporting below.

Now the rest of the source tree is in progress (6 hours now). The first
attempt was about 200MB of files. Now we're importing around 800MB.

After a long period of not using my repository (bad bad bad) I created a
new one today (using the latest Tortoise SVN). Once this was done I
took the easy way out, and set about to import my files.

Perhaps this is important: The source files are located on a linux box
on my local LAN. The box serves as an NT PDC via Samba. The share is
mapped to a local drive. The repository exists on a second linux box,
also on the local LAN. The process of "importing" was to drag and drop
the source directory tree into my new repository. So essentially I'm
going between one linux box to another, via samba on each, through a
Windows XP workstation.

O Joy.

TortoiseSVN 1.3.3, Build 6219 - 32 Bit
Subversion 1.3.1,
apr 0.9.7
apr-iconv 0.9.7
apr-utils 0.9.7
berkeley db 4.3.28
neon 0.25.4
OpenSSL 0.9.8a 11 Oct 2005
zlib 1.2.3

SubVersion is running on a Fedora Core 3 box. I'll call this DEV2:

processor : 0
vendor_id : AuthenticAMD
cpu family : 6
model : 6
model name : AMD Athlon(tm) XP 2000+
stepping : 2
cpu MHz : 1675.772
cache size : 256 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca
cmov pat pse36 mmx fxsr sse syscall mmxext 3dnowext 3dnow
bogomips : 3293.18

The source files are on a Fedora Core 4 box which, as shown below, is a
very capable server (complete with SCSI II Raid). This is called DEV1:

processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 2
model name : Intel(R) XEON(TM) CPU 2.00GHz
stepping : 4
cpu MHz : 1983.484
cache size : 512 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 1
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
bogomips : 3923.96

processor : 1
vendor_id : GenuineIntel
cpu family : 15
model : 2
model name : Intel(R) XEON(TM) CPU 2.00GHz
stepping : 4
cpu MHz : 1983.484
cache size : 512 KB
physical id : 3
siblings : 2
core id : 3
cpu cores : 1
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
bogomips : 3956.73

processor : 2
vendor_id : GenuineIntel
cpu family : 15
model : 2
model name : Intel(R) XEON(TM) CPU 2.00GHz
stepping : 4
cpu MHz : 1983.484
cache size : 512 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 1
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
bogomips : 3956.73

processor : 3
vendor_id : GenuineIntel
cpu family : 15
model : 2
model name : Intel(R) XEON(TM) CPU 2.00GHz
stepping : 4
cpu MHz : 1983.484
cache size : 512 KB
physical id : 3
siblings : 2
core id : 3
cpu cores : 1
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
bogomips : 3956.73

Both machines ought to be fast enough, and there are no other processes
running which take up considerable CPU time. However DEV2 is crawling,
and so the SubVersion repository dialog, *which I'd expect to at least
provide me some feedback*, has been sitting in place for hours and task
manager says it's not responding. That's not accurate. It's not
refreshing. There's a small amount of CPU time being used. 37MB of
ram, however, is being allocated to the (bloated?) process. I know the
process is alive as the window responds to minimize and maximize events.

Oddly (perhaps this can be explained to me in terms of Tortoise or SVN
or both), samba (smbd) is using an enormous amount of CPU resources on
the box running SubVersion:

DEV 2 top:

   PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
19626 root 17 0 12300 4244 9132 R 92.0 0.9 362:36.09 smbd
  3277 root 15 0 46888 8652 37m S 0.3 1.8 11:48.64 X
20414 root 16 0 2260 932 1664 R 0.3 0.2 0:00.07 top
     1 root 16 0 2220 564 1408 S 0.0 0.1 0:01.32 init
     2 root 34 19 0 0 0 S 0.0 0.0 0:00.94 ksoftirqd/0
     3 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 events/0
     4 root 5 -10 0 0 0 S 0.0 0.0 0:00.01 khelper
     5 root 15 -10 0 0 0 S 0.0 0.0 0:00.00 kacpid
    21 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 kblockd/0
    31 root 20 0 0 0 0 S 0.0 0.0 0:00.00 pdflush
    32 root 15 0 0 0 0 S 0.0 0.0 0:00.96 pdflush
    34 root 11 -10 0 0 0 S 0.0 0.0 0:00.00 aio/0
    22 root 15 0 0 0 0 S 0.0 0.0 0:00.00 khubd
    33 root 15 0 0 0 0 S 0.0 0.0 0:13.47 kswapd0
   107 root 25 0 0 0 0 S 0.0 0.0 0:00.00 kseriod
   187 root 7 -10 0 0 0 S 0.0 0.0 0:00.00 kmirrord/0
   198 root 15 0 0 0 0 S 0.0 0.0 0:03.36 kjournald

The box with the source files (the "real" server) is faring a lot better:

DEV 1 top:

   PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
19940 root 15 0 13984 7188 3320 S 4.7 0.7 26:58.41 smbd
25880 root 16 0 2024 996 784 R 0.3 0.1 0:00.05 top
     1 root 16 0 1748 572 492 S 0.0 0.1 0:01.35 init
     2 root RT 0 0 0 0 S 0.0 0.0 0:01.27 migration/0
     3 root 34 19 0 0 0 S 0.0 0.0 0:00.01 ksoftirqd/0
     4 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
     5 root RT 0 0 0 0 S 0.0 0.0 0:02.66 migration/1
     6 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/1
     7 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/1
     8 root RT 0 0 0 0 S 0.0 0.0 0:01.58 migration/2
     9 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/2
    10 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/2
    11 root RT 0 0 0 0 S 0.0 0.0 0:02.30 migration/3
    12 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/3
    13 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/3
    14 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/0
    15 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/1
    16 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/2
    17 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/3
    18 root 11 -5 0 0 0 S 0.0 0.0 0:00.0

So, with all this, any ideas, thoughts, or suggestions?

My sincere thanks for whomever reviews all this and for all the Tortoise
folks who have made a very classy and effective interface!

/m

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tortoisesvn.tigris.org
For additional commands, e-mail: dev-help@tortoisesvn.tigris.org
Received on Thu Jun 1 00:31:56 2006

This is an archived mail posted to the TortoiseSVN Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.