[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Weird server locks up

From: Arturo 'Buanzo' Busleiman <buanzo_at_buanzo.com.ar>
Date: Mon, 30 Jun 2008 19:48:14 -0300

Hash: SHA512

Hi group!

I've found a weird behaviour. First, let me describe my platform:

* An ubuntu server 64bit virtual machine in Vmware Server (Ubuntu Host). The repository is on an
external iSCSI SAN disk, mounted on /srv/svn. (The SAN and LAN networks are physically distinct ones).

* Windows clients using tortoisesvn.

This setup has been working without issues at ALL for the past two weeks (since it was implemented).

Yesterday night, a developer released a bunch of locks, and the next time he attempted to browse the
repository, vmware shouts an "I/O error" on the SAN iSCSI device. From then on, only an "Abort",
then a reboot can get the svn server up and running.

So, I made a full test of every element involved:
* SAN disk was OK, I could read/write data without issues. I even ran, from within the VM guest, an
ext2fs -c. No issues.
* I upgraded all software elements: tortoisesvn, ubuntu guest (except for vmware-server, which was
already 1.0.5 [i know there's 1.0.6, but the changelog shows nothing related).

Then I started active testing:

* run "svn checkout svn://" using the root shell account on the svn server. It
works, no vmware warning window.
* run "svn checkout svn://". Same as before, but using the NIC.
* run "svn checkout svn://" but from another linux machine in the network.
No issues!
* I asked a windows-workstationed developer to download the command-line subversion binaries, and do
~ the same checkout from the same windows machine. It WORKS.

I asked them to test tortoise and BANG, vmware warning. The same warning is triggered when using
tortoisesvn from any other windows computer.

I'm think this has to do with some packet size, data transfer, tortoisesvn-thing while interacting
with a SAN disk (which, again, if I access using standard subversion client, never crashes).

So, to test that theory, I added a standard virtual disk to the guest (a local file on the vm HOST,
not an iSCSI exported lun), rsynced the data from the SAN partition to the new virtual partition...
restarted svn on that new mount point... and tortoisesvn works. no crashes.

Anyone has ANY idea why it might be falling? May I have hit a bug for this kind of scenario? Is
there any other list I should be cross-posting to? My googling keywords involve iscsi, subversion,
vmware, etc. Nothing interesting appears. So, if you can provide any ideas...! :)


- --
Arturo "Buanzo" Busleiman
Independent Security Consultant - SANS - OISSG
Tired of SPAM? Slow Internet in your office? Ask me.

Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org


To unsubscribe, e-mail: users-unsubscribe_at_subversion.tigris.org
For additional commands, e-mail: users-help_at_subversion.tigris.org
Received on 2008-07-01 00:48:42 CEST

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.