[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

[OT] Re: VSS to SVN migration script

From: Nathan Kidd <nathan_at_hummingbird.com>
Date: 2004-05-20 18:41:56 CEST

Toby Johnson wrote:
> Could you send me the entire ss "HISTORY" command for this file that's
> giving you trouble?

[Somewhat off topic.]

A few months ago tried to convert our VSS databases using
both vss2svn.py (https://chidb01.tallan.com:4430/vss2svn/) and
migrate.pl (http://www.riseup.com/~brettw/dev/), without success. (It
seems our databases were much more complex than what most people were
successfully converting.) I spend time hacking vss2svn.py but eventually
gave up and just did a latest-version import, keeping the old VSS
database for history.

My changes to vss2svn.py weren't entirely useless however, and I notice
a few things I solved there cropping up in
http://www.tobiasly.com/svn/vss2svn.pl.

1. Pertinent to this thread, I documented all the possible vss history
outputs I could find. I'm not aware of any missing 'action types'. See
attached vss2svn.py.patch.historydoc.

2. ss.exe output redirected to STDOUT forces hard return wrapping at 80
cols., as noted in the --flatten argument to vss2svn.pl.

" --flatten : "Flatten" migrated comments so that all newlines become
            spaces. If not used, extra "hard" newlines will be inserted
            due to word wrapping of "ss" output; unfortunately there is
            no way to distinguish between "hard" and "soft" newlines
            during migration."

This can be solved by using ss.exe's -o@ option to output to a file and
then read the data from the file. No hard wrapping occurs. This is
particularly important not just for comments: if you have really long
paths in VSS they also will be hard-wrapped! Attached is my patch for
vss2svn.py to do this. (vss2svn.py.patch.nohardwrap)

I made other large changes to vss2svn.py so it understood all the
previously documented history 'action types'. It (the script, not these
particular changes) wasn't in a finished state when I had to move on
("finished" = "converting my database") and the changes are so broad I
don't think they'd be easily accepted as a patch. If someone is
interested in this then drop me a mail.

Regards,

-Nathan

Index: vss2svn.py
===================================================================
--- vss2svn.py (revision 30)
+++ vss2svn.py (working copy)
@@ -448,15 +448,36 @@
                 user_pass = ' -Y' + vss_user + "," + vss_passwd
             else :
                 user_pass = ' -Y' + vss_user
- cmd = '"' + ss + '" History ' + ThisProj + user_pass
- SSPipe = os.popen(cmd)
- history = SSPipe.read()
- retval = SSPipe.close()
+ tempoutputfile = os.path.join(os.environ['TMP'], '~vss2svn-ss-output.tmp')
+ # Must output to file and then read: ss.exe puts hard returns on
+ # wrapped lines when writing to stdout. Stupid!
+
+ # unfortunately ss.exe will only append, so we must delete the
+ # temporary output file first
+ if os.path.exists(tempoutputfile) :
+ retval = os.remove(tempoutputfile)
+ if retval :
+ raise RuntimeError ("Error removing %s: code %d" % (tempoutputfile, retval))
+
+ cmd = '"' + ss + '" History ' + ThisProj + user_pass + ' -o@' + tempoutputfile + " 2>" + tempoutputfile + ".err"
+ retval = os.system(cmd)
+ if retval :
+ SSError = open(tempoutputfile + ".err", "r")
+ errtext = SSError.read()
+ SSError.close()
+ raise RuntimeError ("Error code %d from %s:\n%s" % (retval, cmd, errtext))
+
+ SSOutput = open(tempoutputfile, "rw")
+ history = SSOutput.read()
+ retval = SSOutput.close()
         if retval != None :
             print 'vss command failed:'
             print repr(cmd)
             print history,
             sys.exit(retval)
+
+ os.remove(tempoutputfile);
+
         history2 = history.split('\n*****')
 
         # $/ always gives us problems later on. Chomp off trailing slash

Index: vss2svn.py
===================================================================
--- vss2svn.py (revision 30)
+++ vss2svn.py (working copy)
@@ -392,6 +392,58 @@
 # with VSS (including but not limited to why we must use the command-line
 # tool) read http://www.michaelbolton.net/testing/VSSDefects.html
 
+# We always get history on directories, so all output is from that perspective
+
+# Types of ss history output lines for a directory:
+#--------------------------------------------------
+
+#************ Version 222 ***************** ; first line
+#User: username Date: 3/12/02 Time: 3:48p ; info line, in every rev.
+#Created ; repository created, ver=1
+#Rolled back ; ignore, see file's action
+#FileName.ext added ; file added to directory
+#FileName.ext deleted ; deleted but not destroyed
+#FileName.ext recovered ; undeleted, visible again
+#FileName.ext purged ; deleted item now destroyed
+#FileName.ext destroyed ; item destroyed on delete
+#FileName.ext branched ; works on file level only
+#$/DirName/FileName.ext shared ; closest equivalent is add
+#FileName.ext archived to version XX ; history before XX removed
+#$DirName archived ; essentially, purged
+#$DirName restored ; essentially, added
+#FileName.a renamed to FileName.b ; .a dissapears from history!
+#$DirName.a renamed to $DirName.b ; "
+#$DirName moved to $/new/DirName ; seen in original dir
+#$DirName moved from $/old/DirName ; seen in destination dir
+#;Note: When a file is moved it is seen as separate share and delete actions.
+#Comment: -item 1 fixed ; always last, CR/LF not
+#-item 2 fixed ; escaped: keep reading lines
+
+# Types of ss history output lines for a file object's data:
+#-----------------------------------------------------------
+
+#***** FileName.ext ***** ; first line
+#Version 229 ; revision number
+#User: username Date: 3/12/02 Time: 3:48p ; info line, in every rev.
+#Checked in $/basedir ;
+#Branched ; file dup of dir rev message
+#Rolled back ; ignore, history was purged
+#Comment: -item 1 fixed ; always last, CR/LF not
+#-item 2 fixed ; escaped: keep reading lines
+
+# Types of ss history output lines for a label:
+#---------------------------------------------
+
+#************ Version 222 ***************** ; first line, root of label
+#********************** ; inherited label, ignore
+#Label: "Release Candidate 1" ;
+#User: username Date: 3/12/02 Time: 3:48p ; info line, in every rev
+#Labeled ;
+#Comment: -item 1 fixed ; always last, CR/LF not
+#-item 2 fixed ; escaped: keep reading lines
+#Label comment: ; Parsing note: this will be
+#; eaten by Comment multiline greedyness but is OK. SVN has only 1 comment type
+
 ver_re = re.compile('.*Version (\d+).*')
 user_date_re = re.compile('.*User: (\\S+)\\s+Date:\\s+(\\S+)\\s+Time:\\s+(\\S+).*')
 suffix_action_re = re.compile('^(.+) (added|deleted|purged|destroyed|recovered)$')

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Thu May 20 18:42:29 2004

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.