[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: svnadmin hangs when creating Berkeley repository on NTFS-3G partition

From: Szabolcs Szakacsits <szaka_at_ntfs-3g.org>
Date: Tue, 25 Nov 2008 18:24:59 +0000 (UTC)

Szabolcs Szakacsits <szaka <at> ntfs-3g.org> writes:
> Jaan Vajakas <jaanvajakas <at> hot.ee> writes:
>
> > I have an NTFS partition mounted in Linux at /windows/D. When there is
> > no directory named repos in /windows/D and I run
> > cd /windows/D
> > svnadmin create --fs-type bdb repos
> > then svnadmin just hangs and top shows that most of the CPU is used by
> > mount.ntfs-3g and some CPU is also used by svnadmin (at the time of
> > writing, the TIME+ column in top shows that mount.ntfs-3g has used 50
> > minutes and svnadmin has used 8 minutes CPU time).
> >
> > However, on a reiserfs partition on the same disk, "svnadmin create
> > --fs-type bdb repos" works fine (finishes in a couple of seconds).
> > "svnadmin create repos" (with " --fs-type bdb" omitted) also works
> > fine on NTFS.
> >
> > I am using openSuse 11.0, with the subversion 1.5.2, ntfs-3g 1.5012,
> > fuse 2.7.2-32.1 and kernel 2.6.25.18-0.2-pae.
>
> It takes 0.35 seconds with openSuse 11.0, subversion 1.5.0,
> ntfs-3g 1.5012 (integrated fuse-lite with all FUSE CVS fixes),
> and kernel 2.6.26, x86_64.
>
> # time svnadmin create --fs-type bdb repos
> 0.02s usr, 0.01s sys, 0.35s real, 6% CPU
>
> I'll check subversion 1.5.2 and old kernels if they work differently.

Ok, the issue is a bit subtle.

To make it short, you could upgrade to at least kernel 2.6.26 and
NTFS-3G will work blazing fast: less than a second, like it does
for me.

The long story: the BDB fs-type uses mmap which is supported by all
FUSE based file systems since only 2.6.26 kernels. When this failed
then svnadmin started the cleanup process and this is when things
fell into an infinite loop due to how repos/locks/db.lock is used
in svnadmin and how FUSE handles removed but still open files:

 open repos/locks/db.lock
 ...
 while repos/locks non-empty
    remove all files
 ...
 close repos/locks/db.lock

When a file in use is removed, like db.lock, then the high-level
FUSE API (what currently NTFS-3G also uses) renames the file
to .fuse_hiddenXXXXXXXXXX. The directory won't be empty until
all file handles/references are closed ==> infinite loop. That's
why both NTFS-3G and svnadmin used the CPU.

Potential solutions :

1. svnadmin could(?) use 'if' instead of 'while' above.

2. svnadmin could(?) close the db.lock before the while loop.

3. db.lock could be outside of the repos/locks directory.

4. Some other lock solution (locking itself works absolutely fine
   with all FUSE file systems, even without explicit support).

5. FUSE could be fixed (I very much doubt because the low-level
   interface solves this issue and it would be far from trivial).

6. NTFS-3G could move to the low-level FUSE API which will do
   happen but that will take years.

Btw, this scenario affects all high-level FUSE file systems,
NFS and probably other network file systems too.

Regards, Szaka
 

--
NTFS-3G: http://ntfs-3g.org
 
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe_at_subversion.tigris.org
For additional commands, e-mail: users-help_at_subversion.tigris.org
Received on 2008-11-25 19:25:45 CET

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.