RE: How Big A Dump File Can Be Handled? (svn 1.8 upgrade)

Hi Thomas,

> From: Thomas Harold
> Sent: Friday, 23 August 2013 1:25 AM
> On 8/21/2013 7:13 PM, Geoff Field wrote:
> > I'm keeping the
> > original BDB repositories, with read-only permissions.
> > If I really have the need, I can restart Apache 2 with SVN
> > 1.2.3 and
> > go back to the original repositories....
>
> When we did our 1.6 to 1.8 upgrade a few weeks ago, I used
> the following steps (ours was an in-place upgrade, so a bit
> of extra checking was added):
>
> 0. Back everything up, twice.

Our servers have nightly backups that I know to work (from experience). I also didn't get rid of the originals (as stated).

> 1. Check the version of the repository to see whether it is
> already 1.8

I *knew* that all of our repositories were in the 1.2 format. That's the only version we had for years on end.

...
> 2. Strip permissions on the original repo down to read-only.

 I didn't bother with that, since I didn't do any write operations on the repos (other than changing the names. However, I *did* change the repo access permissions in the authz file.

> 3. Ran "svnadmin verify" on the original repository.

Probably something I should have done, but luckily I ended up with no obvious failures in the dumps.

> 4. Do the "svnadmin dump", piping the output into gzip -5
> (moderate compression).

If you're removing the old repo, I suppose it makes sense to keep the dump file. Compression would make it less onerous in storage terms.

> 5. Remove the old repository directory.

I agree with what the script echoes - "dangerous"

> 6. Create the repository in svn 1.8.

I'm sure there's an "upgrade" command that would do it all in-place.

> 7. Strip permissions on the repository back down to 700,
> owned by root:root while we reload the data.

While, or before?

> 8. Fix the db/fsfs.conf file to take advantage of new features.
>
> Note: Make sure you understand what enable-dir-deltification,
> enable-props-deltification and enable-rep-sharing do. Some
> of these are not turned on in SVN 1.8 by default.

There are features we're very unlikely to need at this stage in our company existence.

> 9. Load the repository back from the dump file.

At last!

> 10. Run "svnadmin pack" to pack revs/revprops files (saves on inodes).

Makes sense

> 11. Run "svnadmin verify".

Always a good thing to do.

> 12. Restore original permissions.

Fair enough.

> Note: I have a custom script that I can run to set
> permissions correctly on our repository directories. I never
> set file system permissions by hand on the repositories, I
> always update the script and then use that.
> (With a few hundred repositories, I have to be organized and rely on
> scripts.)

On your OS, is there a way to read the permissions first?

> 13. Back everything up again, twice.

You're not paranoid if they really *are* out to get you... ;-)

> All-in-all, it took us a few days to convert 110GB of
> repositories (mostly in 1.6 format), but the resulting size
> was only 95GB and far fewer files (due to revprops packing in
> 1.8). Our nightly backup window went from about 3 hours,
> down to 30 minutes from using "svnadmin hotcopy
> --incremental". When then use rdiff-backup to push the
> hotcopy directory to a backup server.

I've just surprised myself by checking the file system properties. After the BDB->FSFS conversion, we now have 164 repositories, totallying 312GB on the disk. That's a LOT of backup space requirement. Luckily for me, that's all handled by our IT department and is done on their SAN via an automatic utility.

Regards,

Geoff

-- 
Apologies for the auto-generated legal boilerplate added by our IT department:
- The contents of this email, and any attachments, are strictly private
and confidential.
- It may contain legally privileged or sensitive information and is intended
solely for the individual or entity to which it is addressed.
- Only the intended recipient may review, reproduce, retransmit, disclose,
disseminate or otherwise use or take action in reliance upon the information
contained in this email and any attachments, with the permission of
Australian Arrow Pty. Ltd.
- If you have received this communication in error, please reply to the sender
immediately and promptly delete the email and attachments, together with
any copies, from all computers.
- It is your responsibility to scan this communication and any attached files
for computer viruses and other defects and we recommend that it be
subjected to your virus checking procedures prior to use.
- Australian Arrow Pty. Ltd. does not accept liability for any loss or damage
of any nature, howsoever caused, which may result
directly or indirectly from this communication or any attached files.