Migrating from CVS to Subversion - Why and How ?

<mailto:users@subversion.tigris.org>

*Why Subversion? *

There are various sites available on Internet discussing how Subversion
is next generation version control system. Here is a decent tabular
write up which sums up everything

http://better-scm.berlios.de/comparison/comparison.html

Bottom line - A linear jump from CVS. But frankly, Open source version
control systems are still lagging. They haven't reached enterprise level
yet.

In a generation, when people expect out of box Single sign-on, easy
integration with existing infrastructure for user and group management ,
web-based administration- subversion still needs to be fine tuned.
Still, it is the best open source version control system available.

*Our existing system - *

We have an employee centric portal with ldap as user data source and an
Identity management tool in place. CVS is our existing version control
system with its own user and group management- cvs-acl.

*Target State : *

Web based user management system with webdav functionality and single
sign-on. Though we had higher ambitions but we re-scoped.

*Hardware : Solaris - 9 x86*

*Solution : *

Subversion provides 3 access mechanisms - being File System based , SVN
Server - custom protocol and last, the best , Web based Apache based
http access. The Subversion book provides enough convincing reasons.

With release 2.0.43 , Apache provides experimental ldap_auth module and
ldap module. ldap auth module as name suggests provides ldap based
authentication mechanism where as mod_ldap provides an elegant way to
access ldap with connection pooling, cache etc.

Another interesting module, mod_dav , for webdav functionality.
Subversion 1.2 provided auto versioning with webdav.

This combination took us close to desired solution. But group management
is still file system based and uses Subversion authorization module

*Software versions : *

- subversion-1.2.0-rc2

- httpd-2.1.3-beta

Beta and RC2 Why? I had confidence on apache server but apache's
Subversion running on 2.1.3 convinced me. Subversion 1.2 has
autoversioning - too tempting :)

*Steps of installation :*

1) Identification and Installation of dependencies

2) Installation of LDAP SDK

3) Build and Installation of Apache webserver

4) Installation and configuration of Subversion

5) Installation of cvs2svn and migration of existing CVS

Base Assumption : GCC , GTK is installed and works

*STEP 1 - Identification and Installation of dependencies*

Set Environment variables PATH and LD_LIBRARY_PATH

LD_LIBRARY_PATH=/usr/local/lib:/usr/lib ; export LD_LIBRARY_PATH

PATH=/usr/sbin:/usr/bin:/usr/local/bin:/usr/ccs/bin:/usr/ucb ; export PATH

a) Expat - C XML parser - Download binaries from Sunfreeware. gunzip and
install

pkgadd -d expat-1.95.5-sol9-intel-local

b) Open SSL - SSL Libraries - Download binaries from Sunfreeware. gunzip
and install

pkgadd -d openssl-0.9.7f-sol9-intel-local

c) GNU sort, which is part of the coreutils package. For CVS to SVN
Conversion

Download coreutils-5.2.1.tar from http://ftp.gnu.org/pub/gnu/coreutils/.

tar xvf coreutils-5.2.1.tar

cd coreutils-5.2.1

./configure ; make ; make install

d) RCS Version 5.7 - Download tar from
http://www.cs.purdue.edu/homes/trinkle/RCS

tar xvf rcs-5.7.tar

cd rcs-5.7

./configure ; make ; make install

e) Libnet for handling of network packets - Download binaries from
Sunfreeware. gunzip and install

pkgadd -d libiconv-1.8-sol9-intel-local

Modify Environment variables

LD_LIBRARY_PATH=/usr/local/lib:/usr/lib:/usr/local/ssl/lib

*STEP 2 - Installation LDAP SDK*

It took me quite a while to get SUN LDAP C SDK binaries for Solaris X86.
Compilation of Mozilla LDAP C SDK version posed some interesting
problems. In search of easy way out, I realized that Directory Server
5.2 resource kit has SUN LDAP C SDK included in it. Download
dsrk52-SunOS5.8_i86pc_OPT.zip from SUN Download site. As per
documentation, Solaris 8 and Solaris 9 use same installer. Create a
temporary folder and unzip the installer

execute "java DSRK". This should install files in desired location.
Don't share libraries with existing LDAP installation.

Again,Modify Environment variables

LD_LIBRARY_PATH=/usr/local/lib:/usr/lib:/usr/local/ssl/lib:/opt/sun/S1DSRK/lib/ldapcsdk/lib:/opt/sun/S1DSRK/lib/nss/lib

STEP3 - Big Step - Build and Installation of Apache webserver

Download httpd-2.1.3-beta.tar from any of apache mirror sites.

tar xvf httpd-2.1.3-beta.tar ; cd httpd-2.1.3-beta

*TIP - ONE *:

configure is a good tool but it can get messy some times. For example -
./configure --help would never give --with-ldap-include as a valid
option but still it is valid option . Configure, most of the time is
hierarchical i.e. invokes configure command of dependent modules.
Execute 'find . -name "configure"' to get list of configure files and
run each of them with --help option to get list of required attributes .
After identifying attributes, add it to top level attributes. The
parameters will be propagated.

*TIP – TWO:*

Enable so is an important option. It ensures that your apache build is
pluggable with new modules at later stages.

*TIP – THREE:*

Apache has something called apache portable runtime and apache portable
runtime utility code named as apr and apr-util. Both apache server and
subversion use this foundation component. Subversion comes with
apr0.96and Apache 2.1.3 with apr 1+ . I did not face any problem with
version mismatch but It took me some time to realize that apr-util
should be built with LDAP support for proper functioning. In case you
use berkely DB as data repository please make sure you have berkley db
options set right.

*TIP – FOUR:*

It is always a good idea to delete code and untar the tar file if build
fails. This stands correct for any code compilation

After many permutations and combinations, I came up with following
configure command

./configure --enable-ldap --enable-authnz-ldap --enable-so \

--enable-authnz-ldap --enable-proxy --enable-ssl --enable-http \

--enable-dav --enable-mods-shared=all \

--with-ldap --enable-ldap --enable-auth-ldap \

--with-ldap-include=/opt/sun/S1DSRK/lib/ldapcsdk/include/ \

--with-ldap-lib=/opt/sun/S1DSRK/lib/ldapcsdk/lib \

--with-ssl=/usr/local/ssl

make

make install // this would install everything under /usr/local/apache2

ls -al /usr/local/apache2/modules to list modules. Make sure you have
all the default + required libraries in the list . Important ones are
mod_authnz_ldap.so, mod_dav.so and mod_ldap.so. Extra one can be plugged
at any time

Open /usr/local/apache2/conf/httpd.conf. Modify following parameters

# User/Group: The name (or #number) of the user/group to run httpd as.

# . On SCO (ODT 3) use "User nouser" and "Group nogroup".

# . On HPUX you may not be able to use shared memory as nobody, and the

# suggested workaround is to create a user www and use that user.

# NOTE that some kernels refuse to setgid(Group) or semctl(IPC_SET)

# when the value of (unsigned)Group is above 60000;

# don't use Group #-1 on these systems!

#

User jesuser

Group jesgroup

#

# Listen: Allows you to bind Apache to specific IP addresses and/or

# ports, instead of the default. See also the <VirtualHost>

# directive.

#

# Change this to Listen on specific IP addresses as shown below to

# prevent Apache from glomming onto all bound IP addresses (0.0.0.0)

#

#Listen 12.34.56.78:80

Listen 5080

# ServerName gives the name and port that the server uses to identify
itself.

# This can often be determined automatically, but we recommend you specify

# it explicitly to prevent problems during startup.

#

# If this is not set to valid DNS name for your host, server-generated

# redirections will not work. See also the UseCanonicalName directive.

#

# If your host doesn't have a registered DNS name, enter its IP address
here.

# You will have to access it by its address anyway, and this will make

# redirections work in a sensible way.

#

ServerName <<>>.icsynergy.<<>>:5080

-----------IMPORTANT-----------

Under Dynamic Shared Object (DSO) Support. mod_ldap should be loaded
before mod_authnz_ldap. The order in which they are loaded is
important.Logical :) but tool me quite a while to figure this one.

LoadModule ldap_module modules/mod_ldap.so

LoadModule authnz_ldap_module modules/mod_authnz_ldap.so

run 'ldd /usr/local/apache2/bin/httpd' to verify if all shared library
requirements are fulfilled. Start webserver for verification.

*STEP 3 - Build and install Subversion *

download subversion-1.2.0-rc2.tar for tigris website

tar xvf subversion-1.2.0-rc2.tar ; cd subversion-1.2.0-rc2

./configure --with-apr=/usr/local/apache2 \

--with-apr-util=/usr/local/apache2

make

make install

execute 'svn' to check if files have been properly installed

ls -al /usr/local/apache2/modules to list modules. mod_dav_svn.so and
mod_authz_svn.so would be two new shared libraries.

Modify /usr/local/apache2/conf/httpd.conf . Verify if following lines
are present.

LoadModule dav_svn_module modules/mod_dav_svn.so

LoadModule authz_svn_module modules/mod_authz_svn.so

Add following location tag

<Location /svn>

# This is root of repository for remote access

DAV svn

# Should be owned by apache user

SVNPath /subversion/icsynergy

SetHandler ldap-status

# Auth type is ldap - very important

AuthBasicProvider ldap

# access only to valid ldap user

require valid-user

AuthType Basic

AuthName "ICSynergy Domain"

AuthLDAPURL
ldap://*****.icsynergy.****:***/ou=People,o=ics,dc=icsynergy,dc=info

AuthLDAPBindDN "***************"

AuthLDAPBindPassword ******

AuthzSVNAccessFile /subversion/AuthzSVNAccessFile

</Location>

create a file /subversion/AuthzSVNAccessFile for authorization purpose

----------------------------------------------
[groups]

developers = uid1, uid2, uid3

[svn:/cvsmodulename/trunk]

@developers = rw
-----------------------

*STEP 5 - Installation of cvs2svn and migration of existing CVS*

Download cvs2svn utility from http://cvs2svn.tigris.org/cvs2svn.html

tar xvf cvs2svn-1.2.1.tar; cd cvs2svn-1.2.1

./configure ; make ; make install

Create a script for migration purpose

###########"cvs2svn.sh"###############

#!/bin/sh

PATH=/usr/sbin:/usr/bin:/usr/local/bin:/usr/ccs/bin:/usr/ucb

MODULE=$1

SVN=/subversion/icsynergy

CVS=/cvs/icsynergy

MSG="re-order repository layout"

# Convert from CVS to SVN using cvs2svn

cvs2svn --existing-svnrepos -s $SVN $CVS/$MODULE

# Then re-order repository layout

svn mkdir file://$SVN/$MODULE -m "$MSG"

svn move file://$SVN/branches file://$SVN/$MODULE/branches -m "$MSG"

svn move file://$SVN/tags file://$SVN/$MODULE/tags -m "$MSG"

svn move file://$SVN/trunk file://$SVN/$MODULE/trunk -m "$MSG"

~

###########"cvs2svn.sh"###############

execute sh cvs2svn.sh for each cvs module.

/Change file permissions/

# chown -R user:group /subversion/icsynergy

# chown -R user:group /usr/local/apache2

# chmod 2775 /subversion/icsynergy/db /subversion/icsynergy/dav

# chmod 0775 /subversion/icsynergy/db/*

Restart Webserver . Now CVS migrated modules can be accessed using
apache as well as locally.

/For remote access/

svn checkout http://****.****.***/svn/cvsmodulename/trunk work-dir

/For local access/

svn checkout file:///subversion/icsynergy/cvsmodulename/trunk work-dir

-- Santosh Haranath

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Thu Apr 28 00:47:10 2005