[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

FSFS Repository corruption on high load on Windows (was Re: svn commit: r1082451 - /subversion/trunk/subversion/libsvn_subr/io.c)

From: Ivan Zhakov <ivan_at_visualsvn.com>
Date: Tue, 14 May 2013 21:06:22 +0400

On Thu, Mar 17, 2011 at 4:21 PM, <rhuijben_at_apache.org> wrote:
> Author: rhuijben
> Date: Thu Mar 17 12:21:29 2011
> New Revision: 1082451
>
> URL: http://svn.apache.org/viewvc?rev=1082451&view=rev
> Log:
> On Windows avoid a file flush that is taking over 15% of the checkout time.
>
> * subversion/libsvn_subr/io.c
> (svn_io_write_unique): Avoid a very costly file flush on Windows.
>

[...]

> --- subversion/trunk/subversion/libsvn_subr/io.c (original)
> +++ subversion/trunk/subversion/libsvn_subr/io.c Thu Mar 17 12:21:29 2011
> @@ -3199,8 +3199,17 @@ svn_io_write_unique(const char **tmp_pat
>
> err = svn_io_file_write_full(new_file, buf, nbytes, NULL, pool);
>
> - if (!err)
> + /* ### BH: Windows doesn't have the race condition between the write and the
> + ### rename that other operating systems might have. So allow windows
> + ### to decide when it wants to perform the disk synchronization using
> + ### the normal file locking and journaling filesystem rules.
> +
> + ### Note that this function doesn't handle the rename, so we aren't even
> + ### sure that we really have to sync. */
> +#ifndef WIN32
> + if (!err && nbytes > 0)
> err = svn_io_file_flush_to_disk(new_file, pool);
> +#endif
>
Hi Bert,

How did you come to conclusion that Windows doesn't have race
condition between write and rename?

It seems Windows actually have race condition between write and move
in case of unexpected computer shutdown. In this case move operation
completes after restart, but file contains zero data. I understand
that you intention was optimize client side operation but
svn_io_write_unique() function actively used in FSFS and lead
repository data corruption.

I've created small program to demonstrate this problem (see
attachment). On start it create four threads actively writing data to
disk and one thread that uses write + move to update file atomically.
If you run this program on virtual machine and then reset then with
very good chance you'll get 'current' file with zeros.

-- 
Ivan Zhakov
CTO | VisualSVN | http://www.visualsvn.com

Received on 2013-05-14 19:07:17 CEST

This is an archived mail posted to the Subversion Dev mailing list.