[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Backing up large repositories

From: J Robert Ray <jrray_at_imageworks.com>
Date: 2004-10-28 03:46:01 CEST

J Robert Ray wrote:
> In svn 1.1.x, the hotcopy code has changed drastically, such that it no
> longer has a hard-coded list of files to copy, but uses readdir/stat to
> (generically) clone an arbitrary directory structure. It can't be
> patched to ignore stat failures and still work correctly.

After studying the code a little more, I realized that it still does
have a hard-coded list of files for bdb repos, but it first has to use
APR to do the 'io_dir_walk' to find all the non-db files to copy. This
is where hotcopy is failing.

I have attached a patch that modifies svn_io_dir_walk so that in the
case where APR returns APR_INCOMPLETE, and the only missing bit of
information is APR_FINFO_TYPE, it assumes the file is a regular file and
not a directory.

With this patch applied, hotcopy succeeds on my large repository. I
feel the patch is relatively safe, the only place I can find where
svn_io_dir_walk is used is when it is called by svn_repos_hotcopy.
Therefore I can reasonably assume that it will never run across a
directory entry larger than 2GB, or fail stating some other special kind
of file, because it only is used to traverse the repository.

Can anybody think of a reason why this patch might be a Very Bad Thing?
  I'm considering rolling it out to my users, so I can upgrade to 1.1.1.

Thanks,

- Robert

Binary files subversion-1.1.1/apr/file_io/unix/.dir.c.swp and subversion-1.1.1-jrray/apr/file_io/unix/.dir.c.swp differ
Binary files subversion-1.1.1/subversion/libsvn_subr/.io.c.swp and subversion-1.1.1-jrray/subversion/libsvn_subr/.io.c.swp differ
diff -rNu subversion-1.1.1/subversion/libsvn_subr/io.c subversion-1.1.1-jrray/subversion/libsvn_subr/io.c
--- subversion-1.1.1/subversion/libsvn_subr/io.c Sat Sep 11 06:27:20 2004
+++ subversion-1.1.1-jrray/subversion/libsvn_subr/io.c Wed Oct 27 18:19:26 2004
@@ -2338,6 +2338,16 @@
       apr_err = apr_dir_read (&finfo, wanted, handle);
       if (APR_STATUS_IS_ENOENT (apr_err))
         break;
+ else if (APR_STATUS_IS_INCOMPLETE (apr_err) && ((finfo.valid & wanted) ^ wanted == APR_FINFO_TYPE))
+ {
+ /* If stat failed because the file is too large, just assume
+ that the file is a regular file and not a directory,
+ it is unlikely to find a directory entry larger than 2GB.
+
+ The only missing bit from finfo.valid allowed is
+ APR_FINFO_TYPE. */
+ finfo.filetype = APR_REG;
+ }
       else if (apr_err)
         {
           return svn_error_wrap_apr

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Thu Oct 28 03:46:40 2004

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.