[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

cvs2svn incremental mode

From: Marko Macek <Marko.Macek_at_gmx.net>
Date: 2002-12-17 21:49:44 CET

Blair Zajac wrote:

>I would think that much of the CVS end of it would be the same. Is this
>true?
>
>Would it be possible to see it now, if that's appropriate? (You may be
>able to tell, I'm anxious to get this other CVS repository tracked :)
>
>

Attached is a quick hack to make cvs2svn work in incremental mode. If
you have the CVS repository available locally (via rsync?) this can do
what you wish.

It adds a --incremental mode which is used after the initial conversion
is done. You need to keep the cvs2svn-data.revs file from the previous
run to work incrementally.

It applies to latest /branches/cvs2svn-mmacek in the subversion repository.

WARNING: only lightly tested, I suspect a few bugs.

A big problem is when something happens (disk full, ^C), there is no way
to recover, you need to start from scratch (create new repository). A
solution to this problem could be saving the CVS revision numbers in
svn properties.

Regards,
Mark

Index: cvs2svn.py
===================================================================
--- cvs2svn.py (revision 4143)
+++ cvs2svn.py (working copy)
@@ -40,6 +40,15 @@
 
 verbose = 1
 
+already_done = { }
+
+def was_done(fname, revision):
+ print "was_done", fname, revision
+ if not already_done.has_key(fname):
+ return 0
+ if not already_done[fname].has_key(revision):
+ return 0
+ return 1
 
 class CollectData(rcsparse.Sink):
   def __init__(self, cvsroot, log_fname_base):
@@ -380,6 +389,22 @@
     # commit this transaction
     print 'committing: %s, over %d seconds' % (time.ctime(self.t_min),
                                                self.t_max - self.t_min)
+ count = 0
+ for f, r, br, tags, branches in self.changes:
+ f = relative_name(ctx.cvsroot, f[:-2])
+ if was_done(f, r):
+ print "skipping", f, r
+ continue
+ count = count + 1
+ for f, r, br, tags, branches in self.deletes:
+ f = relative_name(ctx.cvsroot, f[:-2])
+ if was_done(f, r):
+ print "skipping", f, r
+ continue
+ count = count + 1
+ if count == 0:
+ return
+
 
     if ctx.dry_run:
       for f, r, br, tags, branches in self.changes:
@@ -413,6 +438,9 @@
       # compute a repository path. ensure we have a leading "/" and drop
       # the ,v from the file name
       rel_name = relative_name(ctx.cvsroot, f[:-2])
+ if was_done(rel_name, r):
+ print "skipping", f, r
+ continue
       repos_path = branch_path(ctx, br) + rel_name
       #print 'DEBUG:', repos_path
 
@@ -486,6 +514,9 @@
       # compute a repository path. ensure we have a leading "/" and drop
       # the ,v from the file name
       rel_name = relative_name(ctx.cvsroot, f[:-2])
+ if was_done(rel_name, r):
+ print "skipping", f, r
+ continue
       repos_path = branch_path(ctx, br) + rel_name
 
       print ' deleting %s : %s' % (r, repos_path)
@@ -493,7 +524,8 @@
       # If the file was initially added on a branch, the first mainline
       # revision will be marked dead, and thus, attempts to delete it will
       # fail, since it doesn't really exist.
- if r != '1.1':
+ #if r != '1.1':
+ if fs.check_path(root, repos_path, f_pool) != util.svn_node_none:
         ### need to discriminate between OS paths and FS paths
         fs.delete(root, repos_path, f_pool)
 
@@ -621,6 +653,18 @@
     output.write('%s ' % (branch));
   output.write('%s\n' % fname);
 
+def pass0(ctx):
+ if ctx.incremental:
+ for line in fileinput.FileInput(ctx.log_fname_base + REVS_SUFFIX):
+ timestamp, digest, op, rev, fname, branch_name, tags, branches = \
+ parse_revs_line(line)
+ print fname
+ fname = relative_name(ctx.cvsroot, fname[:-2]);
+ if not already_done.has_key(fname):
+ already_done[fname] = { }
+ already_done[fname][rev] = 1
+ print "DONE", already_done;
+
 def pass1(ctx):
   cd = CollectData(ctx.cvsroot, DATAFILE)
   p = rcsparse.Parser()
@@ -737,6 +781,7 @@
     print count, 'commits processed.'
 
 _passes = [
+ pass0,
   pass1,
   pass2,
   pass3,
@@ -776,6 +821,7 @@
   print ' --branches=PATH path for branches (default: %s)' % ctx.branches_base
   print ' --tags=PATH path for tags (default: %s)' % ctx.tags_base
   print ' --encoding=ENC encoding of log messages in CVS repos (default: %s)' % ctx.encoding
+ print ' --incremental read .revs file and process only new files/revisions'
   sys.exit(1)
 
 def main():
@@ -790,11 +836,12 @@
   ctx.trunk_base = "/trunk"
   ctx.tags_base = "/tags"
   ctx.branches_base = "/branches"
- ctx.encoding = "ascii"
+ ctx.encoding = "iso-8859-1"
+ ctx.incremental = 0
 
   try:
     opts, args = getopt.getopt(sys.argv[1:], 'p:s:vn',
- [ "create", "trunk=", "branches=", "tags=", "encoding=" ])
+ [ "create", "trunk=", "branches=", "tags=", "encoding=", "incremental" ])
   except getopt.GetoptError:
     usage(ctx)
   if len(args) != 1:
@@ -826,6 +873,8 @@
       ctx.tags_base = value
     elif opt == '--encoding':
       ctx.encoding = value
+ elif opt == '--incremental':
+ ctx.incremental = 1
 
   util.run_app(convert, ctx, start_pass=start_pass)
 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Tue Dec 17 21:47:36 2002

This is an archived mail posted to the Subversion Dev mailing list.