Great to see this!
Thank you, Eric.
On Thu, Oct 8, 2009 at 17:44, Eric S. Raymond <esr_at_thyrsus.com> wrote:
> Author: esr
> Date: Thu Oct  8 14:44:00 2009
> New Revision: 39884
>
> Log:
> Initial commit of svncutter.
>
> Added:
> Â trunk/contrib/server-side/svncutter/
> Â trunk/contrib/server-side/svncutter/README
>  trunk/contrib/server-side/svncutter/svncutter  (contents, props changed)
> Modified:
> Â trunk/COMMITTERS
>
> Modified: trunk/COMMITTERS
> URL: http://svn.collab.net/viewvc/svn/trunk/COMMITTERS?pathrev=39884&r1=39883&r2=39884
> ==============================================================================
> --- trunk/COMMITTERS   Thu Oct  8 12:47:36 2009     (r39883)
> +++ trunk/COMMITTERS   Thu Oct  8 14:44:00 2009     (r39884)
> @@ -138,6 +138,7 @@ Commit access for specific areas:
>      nmiyo  MIYOKAWA, Nobuyoshi <n-miyo_at_tempus.org>   (www: ja)
>     rocksun  Rock Sun <daijun_at_gmail.com>         (www: zh)
>     kmradke  Kevin Radke <kmradke_at_gmail.com>       (add-needs-lock.py)
> +      esr  Eric S. Raymond <esr_at_thyrsus.com>      (svncutter)
>
> Â Translation of message files:
>
>
> Added: trunk/contrib/server-side/svncutter/README
> URL: http://svn.collab.net/viewvc/svn/trunk/contrib/server-side/svncutter/README?pathrev=39884
> ==============================================================================
> --- /dev/null  00:00:00 1970  (empty, because file is newly added)
> +++ trunk/contrib/server-side/svncutter/README  Thu Oct  8 14:44:00 2009     (r39884)
> @@ -0,0 +1,5 @@
> +svncutter is a tool for performing various surgical operations on
> +Subversion dump files.
> +
> +This directory exists to hold svncutter and some unit tests that don't exist
> +yet.
>
> Added: trunk/contrib/server-side/svncutter/svncutter
> URL: http://svn.collab.net/viewvc/svn/trunk/contrib/server-side/svncutter/svncutter?pathrev=39884
> ==============================================================================
> --- /dev/null  00:00:00 1970  (empty, because file is newly added)
> +++ trunk/contrib/server-side/svncutter/svncutter    Thu Oct  8 14:44:00 2009     (r39884)
> @@ -0,0 +1,692 @@
> +#!/usr/bin/env python
> +#
> +# Hacked together by ESR, October 2009. New BSD license applies.
> +# The Subversion project is explicitly granted permission to redistribute
> +# under the prevailing license of their project.
> +
> +"""
> +svncutter - clique-squash, range-selection, and property mutations on SVN dump files
> +general usage: svncutter [-q] [-r SELECTION] SUBCOMMAND
> +
> +In all commands, the -r (or --range) option limits the selection of revisions
> +over which an operation will be performed. A selection consists of
> +one or more comma-separated ranges. A range may consist of an integer
> +revision number or the special name HEAD for the head revision. Or it
> +may be a colon-separated pair of integers, ir an integer followed by a
> +colon followed by HEAD.
> +
> +Normally, each subcommand produces a progress spinner on standard
> +error; each turn means another revision has been filtered. The -q (or
> +--quiet) option suppresses this.
> +
> +Type 'svncutter help <subcommand>' for help on a specific subcommand.
> +
> +Available subcommands:
> + Â squash
> + Â select
> + Â propdel
> + Â propset
> + Â proprename
> + Â log
> + Â setlog
> +"""
> +
> +oneliners = {
> + Â Â "squash": Â Â "Squashing revisions",
> + Â Â "select": Â Â "Selecting revisions",
> + Â Â "propdel": Â Â "Deleting revision properties",
> + Â Â "propset": Â Â "Setting revision properties",
> + Â Â "proprename": "Renaming revision properties",
> + Â Â "log": Â Â Â Â "Extracting log entries",
> + Â Â "setlog": Â Â "Mutating log entries",
> + Â Â }
> +
> +helpdict = {
> + Â Â "squash": """\
> +squash: usage: svncutter [-q] [-r SELECTION] [-m mapfile] [-f] [-c] squash
> +
> +The 'squash' subcommand merges adjacent commits that have the same
> +author and log text and were made within 5 minutes of each other.
> +This can be helpful in cleaning up after migrations from file-oriented
> +revision control systems, or if a developer has been using a pre-2006
> +version of Emacs VC.
> +
> +With the -m (or --mapfile) option, squash emits a map to tne named
> +file showing how old revision numbers map into new ones.
> +
> +With the -e (or --excise) option, the specified set of revisions in
> +unconditionally removed. Â The tool will exit with an error if an
> +excised remove is part of a clique eligible for squashing. Â Note that
> +svncutter does not perform any checks on whether the repository
> +history is afterwards valid; if you delete a node using this option,
> +you won't find out you have a problem intil you attempt to load the
> +resulting dumpfile.
> +
> +svncutter attempts to fix up references to Subversion revisions in log
> +entries so they will still be correct after squashing. Â It considers
> +anything that looks like the regular expression \\br[0-9]+\\b to be
> +a comment reference (this is the same format that Subversion uses
> +in log headers).
> +
> +Every revision in the file after the first omiited onf gets the property
> +'svncutter:original' set to the revision number it had before the
> +squash operation.
> +
> +The option --f (or --flagrefs) causes svncutter to wrap its revision-reference
> +substitutions in curly braces ({}). Â By doing this, then grepping for 'r{'
> +in the output of 'svncutter log', you can check for false conversions.
> +
> +The -c (or --compressmap) option changes the mapfile format to one
> +that is easier for human browsing, though less well suited for
> +interpretation by other programs.
> +""",
> + Â Â "select": """\
> +select: usage: svncutter [-q] [-r SELECTION] select
> +
> +The 'select' subcommand selects a range and permits only revisions in
> +that range to pass to standard output. Â A range beginning with 0
> +includes the dumpfile header.
> +""",
> + Â Â "propdel": """\
> +propdel: usage: svncutter ---revprop PROPNAME [-r SELECTION] propdel
> +
> +Delete the unversioned revision property PROPNAME. May
> +be restricted by a revision selection. You may specify multiple
> +prperties to be deleted.
> +""",
> + Â Â "propset": """\
> +propset: usage: svncutter ---revprop PROPNAME=PROPVAL [-r SELECTION] propset
> +
> +Set the unversioned revision property PROPNAME to PROPVAL. May
> +be restricted by a revision selection. You may specify multiple
> +prperties to be deleted.
> +""",
> + Â Â "proprename": """\
> +proprename: usage: svncutter ---revprop OLDNAME->NEWNAME [-r SELECTION] proprename
> +
> +Rename the unversioned revision property OLDNAME to NEWNAME. May
> +be restricted by a revision selection. You may specify multiple
> +prperties to be renamed.
> +""",
> + Â Â "log": """\
> +log: usage: svncutter [-r SELECTION] log
> +
> +Generate a log report, same format as the output of svn log on a
> +repository, to standard output.
> +""",
> + Â Â "setlog": """\
> +setlog: usage: svncutter [-r SELECTION] --logentries=LOGFILE setlog
> +
> +Replace the log entries in the input dumpfile with the corresponding entries
> +in the LOGFILE, which should be in the format of an svn log output.
> +Replacements may be restricted to a specified range.
> +""",
> + Â Â }
> +
> +import os, sys, calendar, time, getopt, re
> +
> +class Baton:
> + Â Â "Ship progress indications to stderr."
> + Â Â def __init__(self, prompt, endmsg=None):
> + Â Â Â Â self.stream = sys.stderr
> + Â Â Â Â self.stream.write(prompt + "...")
> + Â Â Â Â if os.isatty(self.stream.fileno()):
> + Â Â Â Â Â Â self.stream.write(" \010")
> + Â Â Â Â self.stream.flush()
> + Â Â Â Â self.count = 0
> + Â Â Â Â self.endmsg = endmsg
> + Â Â Â Â self.time = time.time()
> + Â Â Â Â return
> +
> + Â Â def twirl(self, ch=None):
> + Â Â Â Â if self.stream is None:
> + Â Â Â Â Â Â return
> + Â Â Â Â if os.isatty(self.stream.fileno()):
> + Â Â Â Â Â Â if ch:
> + Â Â Â Â Â Â Â Â self.stream.write(ch)
> + Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â self.stream.write("-/|\\"[self.count % 4])
> + Â Â Â Â Â Â Â Â self.stream.write("\010")
> + Â Â Â Â Â Â self.stream.flush()
> + Â Â Â Â self.count = self.count + 1
> + Â Â Â Â return
> +
> + Â Â def end(self, msg=None):
> + Â Â Â Â if msg == None:
> + Â Â Â Â Â Â msg = self.endmsg
> + Â Â Â Â if self.stream:
> + Â Â Â Â Â Â self.stream.write(("...(%2.2f sec) %s." % (time.time() - self.time, msg)) + os.linesep)
> + Â Â Â Â return
> +
> +class LineBufferedSource:
> + Â Â "Generic class for line-buffered input with pushback."
> + Â Â def __init__(self, infile):
> + Â Â Â Â self.linebuffer = None
> + Â Â Â Â self.file = infile
> + Â Â Â Â self.linenumber = 0
> + Â Â def readline(self):
> + Â Â Â Â "Line-buffered readline."
> + Â Â Â Â if self.linebuffer:
> + Â Â Â Â Â Â line = self.linebuffer
> + Â Â Â Â Â Â self.linebuffer = None
> + Â Â Â Â else:
> + Â Â Â Â Â Â line = self.file.readline()
> + Â Â Â Â Â Â self.linenumber += 1
> + Â Â Â Â return line
> + Â Â def require(self, prefix):
> + Â Â Â Â "Read a line, require it to have a specified prefix."
> + Â Â Â Â line = self.readline()
> + Â Â Â Â if not line:
> + Â Â Â Â Â Â sys.stderr.write("svncutter: unexpected end of input." + os.linesep)
> + Â Â Â Â Â Â sys.exit(1)
> + Â Â Â Â assert line.startswith(prefix)
> + Â Â Â Â return line
> + Â Â def read(self, len):
> + Â Â Â Â "Straight read from underlying file, no buffering."
> + Â Â Â Â assert(self.linebuffer is None)
> + Â Â Â Â text = self.file.read(len)
> + Â Â Â Â self.linenumber += text.count(os.linesep[0])
> + Â Â Â Â return text
> + Â Â def peek(self):
> + Â Â Â Â "Peek at the next line in the source."
> + Â Â Â Â assert(self.linebuffer is None)
> + Â Â Â Â self.linebuffer = self.file.readline()
> + Â Â Â Â return self.linebuffer
> + Â Â def flush(self):
> + Â Â Â Â "Get the contents of the line buffer, clearing it."
> + Â Â Â Â assert(self.linebuffer is not None)
> + Â Â Â Â line = self.linebuffer
> + Â Â Â Â self.linebuffer = None
> + Â Â Â Â return line
> + Â Â def push(self, line):
> + Â Â Â Â "Push a line back to the line buffer."
> + Â Â Â Â assert(self.linebuffer is None)
> + Â Â Â Â self.linebuffer = line
> + Â Â def has_line_buffered(self):
> + Â Â Â Â return self.linebuffer is not None
> +
> +class DumpfileSource(LineBufferedSource):
> + Â Â "This class knows about dumpfile format."
> + Â Â def __init__(self, infile, baton=None):
> + Â Â Â Â LineBufferedSource.__init__(self, infile)
> + Â Â Â Â self.baton = baton
> + Â Â def read_revision_header(self, property_hook=None):
> + Â Â Â Â "Read a revision header, parsing its proprties."
> + Â Â Â Â properties = {}
> + Â Â Â Â propkeys = []
> + Â Â Â Â stash = self.require("Revision-number:")
> + Â Â Â Â revision = int(stash.split()[1])
> + Â Â Â Â stash += self.require("Prop-content-length:")
> + Â Â Â Â stash += self.require("Content-length:")
> + Â Â Â Â stash += self.require(os.linesep)
> + Â Â Â Â while not self.peek().startswith("PROPS-END"):
> + Â Â Â Â Â Â self.require("K")
> + Â Â Â Â Â Â keyhd = self.readline()
> + Â Â Â Â Â Â key = keyhd.strip()
> + Â Â Â Â Â Â valhd = self.require("V")
> + Â Â Â Â Â Â vlen = int(valhd.split()[1])
> + Â Â Â Â Â Â value = self.read(vlen)
> + Â Â Â Â Â Â self.require(os.linesep)
> + Â Â Â Â Â Â properties[key] = value
> + Â Â Â Â Â Â propkeys.append(key)
> + Â Â Â Â if property_hook:
> + Â Â Â Â Â Â (propkeys, properties) = property_hook(propkeys, properties, revision)
> + Â Â Â Â for key in propkeys:
> + Â Â Â Â Â Â if key in properties:
> + Â Â Â Â Â Â Â Â stash += "K %d%s" % (len(key), os.linesep)
> + Â Â Â Â Â Â Â Â stash += "%s%s" % (key, os.linesep)
> + Â Â Â Â Â Â Â Â stash += "V %d%s" % (len(properties[key]), os.linesep)
> + Â Â Â Â Â Â Â Â stash += "%s%s" % (properties[key], os.linesep)
> + Â Â Â Â stash += self.flush()
> + Â Â Â Â if self.baton:
> + Â Â Â Â Â Â self.baton.twirl()
> + Â Â Â Â return (revision, stash, properties)
> + Â Â def read_until_next(self, prefix, revmap=None):
> + Â Â Â Â "Accumulate lines until the next matches a specified prefix."
> + Â Â Â Â stash = ""
> + Â Â Â Â while True:
> + Â Â Â Â Â Â line = self.readline()
> + Â Â Â Â Â Â if not line:
> + Â Â Â Â Â Â Â Â return stash
> + Â Â Â Â Â Â elif line.startswith(prefix):
> + Â Â Â Â Â Â Â Â self.push(line)
> + Â Â Â Â Â Â Â Â return stash
> + Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â # Hack the revision levels in copy-from headers.
> + Â Â Â Â Â Â Â Â # We're actually modifying the dumpfile contents
> + Â Â Â Â Â Â Â Â # (rather than selectively omitting parts of it).
> + Â Â Â Â Â Â Â Â # Note: this will break on a dumpfile that has dumpfiles
> + Â Â Â Â Â Â Â Â # in its nodes!
> + Â Â Â Â Â Â Â Â if revmap and line.startswith("Node-copyfrom-rev:"):
> + Â Â Â Â Â Â Â Â Â Â oldrev = line.split()[1]
> + Â Â Â Â Â Â Â Â Â Â line = line.replace(oldrev, `revmap[int(oldrev)]`)
> + Â Â Â Â Â Â Â Â stash += line
> + Â Â def apply_property_hook(self, selection, hook):
> + Â Â Â Â "Apply a property transformation on a specified range."
> + Â Â Â Â def innerhook(keyprops, propdict, revision):
> + Â Â Â Â Â Â if revision in selection:
> + Â Â Â Â Â Â Â Â return hook(keyprops, propdict, revision)
> + Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â return (keyprops, propdict)
> + Â Â Â Â while True:
> + Â Â Â Â Â Â sys.stdout.write(self.read_until_next("Revision-number:"))
> + Â Â Â Â Â Â if not self.has_line_buffered():
> + Â Â Â Â Â Â Â Â return
> + Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â (revision,stash,properties) = self.read_revision_header(innerhook)
> + Â Â Â Â Â Â Â Â sys.stdout.write(stash)
> +
> + Â Â def __del__(self):
> + Â Â Â Â if self.baton:
> + Â Â Â Â Â Â self.baton.end()
> +
> +class SubversionRange:
> + Â Â def __init__(self, txt):
> + Â Â Â Â self.txt = txt
> + Â Â Â Â self.intervals = []
> + Â Â Â Â for (i, item) in enumerate(txt.split(",")):
> + Â Â Â Â Â Â if ':' in item:
> + Â Â Â Â Â Â Â Â (lower, upper) = item.split(':')
> + Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â lower = upper = item
> + Â Â Â Â Â Â if lower.isdigit():
> + Â Â Â Â Â Â Â Â lower = int(lower)
> + Â Â Â Â Â Â if upper.isdigit():
> + Â Â Â Â Â Â Â Â upper = int(upper)
> + Â Â Â Â Â Â self.intervals.append((lower, upper))
> + Â Â def __contains__(self, rev):
> + Â Â Â Â for (lower, upper) in self.intervals:
> + Â Â Â Â Â Â if lower == "HEAD":
> + Â Â Â Â Â Â Â Â sys.stderr.write("svncutter: can't accept HEAD as lower bound of a range.\n")
> + Â Â Â Â Â Â Â Â sys.exit(1)
> + Â Â Â Â Â Â elif upper == "HEAD" or rev in range(lower, upper+1):
> + Â Â Â Â Â Â Â Â return True
> + Â Â Â Â return False
> + Â Â def upperbound(self):
> + Â Â Â Â "What is the uppermost revision in the spec?"
> + Â Â Â Â if self.intervals[-1][1] == "HEAD":
> + Â Â Â Â Â Â return sys.maxint
> + Â Â Â Â else:
> + Â Â Â Â Â Â return self.intervals[-1][1]
> + Â Â def __repr__(self):
> + Â Â Â Â return self.txt
> +
> +class Logfile:
> + Â Â "Represent the state of a lofile"
> + Â Â def __init__(self, readable, restriction=None):
> + Â Â Â Â self.comments = {}
> + Â Â Â Â self.source = LineBufferedSource(readable)
> + Â Â Â Â state = 'awaiting_header'
> + Â Â Â Â author = date = None
> + Â Â Â Â logentry = ""
> + Â Â Â Â lineno = 0
> + Â Â Â Â while True:
> + Â Â Â Â Â Â lineno += 1
> + Â Â Â Â Â Â line = readable.readline()
> + Â Â Â Â Â Â if state == 'in_logentry':
> + Â Â Â Â Â Â Â Â if not line or line.startswith("-----------"):
> + Â Â Â Â Â Â Â Â Â Â if rev:
> + Â Â Â Â Â Â Â Â Â Â Â Â logentry = logentry.strip()
> + Â Â Â Â Â Â Â Â Â Â Â Â if restriction is None or rev in restriction:
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â self.comments[rev] = (author, date, logentry)
> + Â Â Â Â Â Â Â Â Â Â Â Â rev = None
> + Â Â Â Â Â Â Â Â Â Â Â Â logentry = ""
> + Â Â Â Â Â Â Â Â Â Â if line:
> + Â Â Â Â Â Â Â Â Â Â Â Â state = 'awaiting_header'
> + Â Â Â Â Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â Â Â Â Â break
> + Â Â Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â Â Â logentry += line
> + Â Â Â Â Â Â elif state == 'awaiting_header':
> + Â Â Â Â Â Â Â Â if not line:
> + Â Â Â Â Â Â Â Â Â Â break
> + Â Â Â Â Â Â Â Â elif line.startswith("-----------"):
> + Â Â Â Â Â Â Â Â Â Â continue
> + Â Â Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â Â Â m = re.match("r[0-9]+", line)
> + Â Â Â Â Â Â Â Â Â Â if not m:
> + Â Â Â Â Â Â Â Â Â Â Â Â sys.stderr.write('"%s", line %d: svncutter did not see a comment header where one was expected\n' % (readable.name, lineno))
> + Â Â Â Â Â Â Â Â Â Â Â Â sys.exit(1)
> + Â Â Â Â Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â Â Â Â Â fields = line.split("|")
> + Â Â Â Â Â Â Â Â Â Â Â Â (rev, author, date, linecount) = map(lambda x: x.strip(), fields)
> + Â Â Â Â Â Â Â Â Â Â Â Â rev = rev[1:] Â # strip off leaing 'r'
> + Â Â Â Â Â Â Â Â Â Â Â Â state = 'in_logentry'
> +
> + Â Â def __contains__(self, key):
> + Â Â Â Â return str(key) in self.comments
> + Â Â def __getitem__(self, key):
> + Â Â Â Â "Emulate dictionary, for new-style interface."
> + Â Â Â Â return self.comments[str(key)]
> +
> +def isotime(s):
> + Â Â "ISO 8601 to local clock time."
> + Â Â if s[-1] == "Z":
> + Â Â Â Â s = s[:-1]
> + Â Â if "." in s:
> + Â Â Â Â (date, msec) = s.split(".")
> + Â Â else:
> + Â Â Â Â date = s
> + Â Â Â Â msec = "0"
> + Â Â # Note: no leap-second correction!
> + Â Â return calendar.timegm(time.strptime(date, "%Y-%m-%dT%H:%M:%S")) + float("0." + msec)
> +
> +def reference_mapper(value, mutator, flagrefs=False):
> + Â Â "Apply a mutator function to revision references."
> + Â Â revrefs = []
> + Â Â for matchobj in re.finditer(r'\br([0-9]+)\b', value):
> + Â Â Â Â revrefs.append(matchobj)
> + Â Â if revrefs:
> + Â Â Â Â revrefs.reverse()
> + Â Â Â Â for m in revrefs:
> + Â Â Â Â Â Â new = mutator(m.group(1))
> + Â Â Â Â Â Â if flagrefs:
> + Â Â Â Â Â Â Â Â new = "{" + new + "}"
> + Â Â Â Â Â Â if new != m.group(1):
> + Â Â Â Â Â Â Â Â value = value[:m.start(1)] + new + value[m.end(1):]
> + Â Â return value
> +
> +# Generic machinery ends here, actual command implementations begin
> +
> +def squash(source, timefuzz,
> + Â Â Â Â Â mapto=None, selection=None, excise=None,
> + Â Â Â Â Â flagrefs=False, compressmap=False):
> + Â Â "Coalesce adjacent commits with same author+log and close timestamps."
> + Â Â dupes = []
> + Â Â # The tricky bit is rewriting the revision numbers in node headers
> + Â Â # associated with copy actions.
> + Â Â clique_map = {} Â Â # Map revisions to the base reves of their cliques
> + Â Â squash_map = {} Â Â # Map clique bases revs to their squashed numbers
> + Â Â skipcount = numbered = clique_base = 0
> + Â Â outmap = []
> + Â Â def hacklog(propkeys, propdict, revision):
> + Â Â Â Â # Hack references to revision levels in comments.
> + Â Â Â Â for (key, value) in propdict.items():
> + Â Â Â Â Â Â if key == "svn:log":
> + Â Â Â Â Â Â Â Â propdict[key] = reference_mapper(value, lambda old: str(squash_map[clique_map[int(old)]]), flagrefs)
> + Â Â Â Â return (propkeys, propdict)
> + Â Â prevprops = {"svn:log":"", "svn:author":"", "svn:date":0}
> + Â Â omit = excise is not None and 0 in excise
> + Â Â while True:
> + Â Â Â Â stash = source.read_until_next("Revision-number:", clique_map)
> + Â Â Â Â if not omit:
> + Â Â Â Â Â Â sys.stdout.write(stash)
> + Â Â Â Â if not source.has_line_buffered():
> + Â Â Â Â Â Â if excise is not None and dupes and dupes[0] in excise:
> + Â Â Â Â Â Â Â Â outmap.append((None, dupes))
> + Â Â Â Â Â Â elif numbered >= 1:
> + Â Â Â Â Â Â Â Â outmap.append((numbered-1, dupes))
> + Â Â Â Â Â Â break
> + Â Â Â Â else:
> + Â Â Â Â Â Â (revision, stash, properties) = source.read_revision_header(hacklog)
> + Â Â Â Â Â Â # We have all properties of this revision.
> + Â Â Â Â Â Â # Compute whether to merge it with the previous one.
> + Â Â Â Â Â Â skip = "svn:log" in properties and "svn:author" in properties \
> + Â Â Â Â Â Â Â Â Â and properties["svn:log"] == prevprops.get("svn:log") \
> + Â Â Â Â Â Â Â Â Â and properties["svn:author"] == prevprops.get("svn:author") \
> + Â Â Â Â Â Â Â Â Â and (selection is None or revision in selection) \
> + Â Â Â Â Â Â Â Â Â and abs(isotime(properties["svn:date"]) - isotime(prevprops.get("svn:date"))) < timefuzz
> + Â Â Â Â Â Â # Did user request an unconditional omission?
> + Â Â Â Â Â Â omit = excise is not None and revision in excise
> + Â Â Â Â Â Â if skip and omit:
> + Â Â Â Â Â Â Â Â sys.stderr.write("squash: can't omit a revision about to be squashed.\n")
> + Â Â Â Â Â Â Â Â sys.exit(1)
> + Â Â Â Â Â Â # Treat spans of omitted commits as cliques for reporting
> + Â Â Â Â Â Â if omit and excise is not None and revision-1 in excise:
> + Â Â Â Â Â Â Â Â skip = True
> + Â Â Â Â Â Â # The magic moment
> + Â Â Â Â Â Â if skip:
> + Â Â Â Â Â Â Â Â skipcount += 1
> + Â Â Â Â Â Â Â Â clique_map[revision] = clique_base
> + Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â clique_base = revision
> + Â Â Â Â Â Â Â Â clique_map[clique_base] = clique_base
> + Â Â Â Â Â Â Â Â squash_map[clique_base] = revision - skipcount
> + Â Â Â Â Â Â Â Â if excise is not None and dupes and dupes[0] in excise:
> + Â Â Â Â Â Â Â Â Â Â outmap.append((None, dupes))
> + Â Â Â Â Â Â Â Â elif numbered >= 1:
> + Â Â Â Â Â Â Â Â Â Â outmap.append((numbered-1, dupes))
> + Â Â Â Â Â Â Â Â dupes = []
> + Â Â Â Â Â Â Â Â if omit:
> + Â Â Â Â Â Â Â Â Â Â skipcount += 1
> + Â Â Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â Â Â sys.stdout.write(stash)
> + Â Â Â Â Â Â Â Â Â Â prevprops = properties
> + Â Â Â Â Â Â Â Â Â Â numbered += 1
> + Â Â Â Â Â Â dupes.append(revision)
> + Â Â Â Â # Go back around to copying to the next revision header.
> + Â Â if mapto:
> + Â Â Â Â mapto.write(("%% %d out of %d original revisions squashed, leaving %d" \
> + Â Â Â Â Â Â Â Â Â Â % (skipcount, revision, numbered-1)) + os.linesep)
> + Â Â Â Â if not compressmap:
> + Â Â Â Â Â Â for (numbered, dupes) in outmap:
> + Â Â Â Â Â Â Â Â if numbered is None:
> + Â Â Â Â Â Â Â Â Â Â mapto.write(" Â None <- " + " ".join(map(str, dupes))+os.linesep)
> + Â Â Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â Â Â mapto.write(("%6d <- " % numbered) + " ".join(map(str, dupes))+os.linesep)
> + Â Â Â Â else:
> + Â Â Â Â Â Â compressed = []
> + Â Â Â Â Â Â force_new_range = True
> + Â Â Â Â Â Â last_n = -1
> + Â Â Â Â Â Â last_oldrevs = []
> + Â Â Â Â Â Â # Process the raw outmap into a form that compressees ranges.
> + Â Â Â Â Â Â # Squash cliques are left alone. Â Ranger between
> + Â Â Â Â Â Â # them map to either
> + Â Â Â Â Â Â # (1) None followed by a singleton list (single deleted rev)
> + Â Â Â Â Â Â # (2) None followed by a two-element list (range of deletions)
> + Â Â Â Â Â Â # (3) Single number followed by singleton list = 1-element range)
> + Â Â Â Â Â Â # (4) Two-element list followed by two-element list =
> + Â Â Â Â Â Â # Â Â multiple elements, old range to new range.
> + Â Â Â Â Â Â for (n, oldrevs) in outmap:
> + Â Â Â Â Â Â Â Â #print >>sys.stderr, "I see:", (n, oldrevs)
> + Â Â Â Â Â Â Â Â cliquebase = oldrevs[0]
> + Â Â Â Â Â Â Â Â if len(oldrevs) > 1:
> + Â Â Â Â Â Â Â Â Â Â compressed.append((n, oldrevs))
> + Â Â Â Â Â Â Â Â Â Â force_new_range = True
> + Â Â Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â Â Â if (n is None) != (last_n is None):
> + Â Â Â Â Â Â Â Â Â Â Â Â #print >>sys.stderr, "Forcing range break"
> + Â Â Â Â Â Â Â Â Â Â Â Â force_new_range = True
> + Â Â Â Â Â Â Â Â Â Â if force_new_range:
> + Â Â Â Â Â Â Â Â Â Â Â Â compressed.append((n, oldrevs))
> + Â Â Â Â Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â Â Â Â Â #print >>sys.stderr, "Adding to range"
> + Â Â Â Â Â Â Â Â Â Â Â Â if len(last_oldrevs) == 1:
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â oldrevs = last_oldrevs + oldrevs
> + Â Â Â Â Â Â Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â oldrevs = last_oldrevs[:1] + oldrevs
> + Â Â Â Â Â Â Â Â Â Â Â Â lowerbound = compressed[-1][0]
> + Â Â Â Â Â Â Â Â Â Â Â Â if (last_n is None) and (n is None):
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â compressed[-1] = [None, oldrevs]
> + Â Â Â Â Â Â Â Â Â Â Â Â elif type(lowerbound) == type(0):
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â compressed[-1] = [[lowerbound, n], oldrevs]
> + Â Â Â Â Â Â Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â compressed[-1] = [lowerbound[:1] + [n], oldrevs]
> + Â Â Â Â Â Â Â Â Â Â force_new_range = False
> + Â Â Â Â Â Â Â Â Â Â last_n = n
> + Â Â Â Â Â Â Â Â Â Â last_oldrevs = oldrevs
> + Â Â Â Â Â Â #print >>sys.stderr, "Compressed:", compressed
> + Â Â Â Â Â Â for (a, b) in compressed:
> + Â Â Â Â Â Â Â Â if a is None:
> + Â Â Â Â Â Â Â Â Â Â if len(b) == 1:
> +             print >>mapto, "  None     <- %d" % b[0]
> + Â Â Â Â Â Â Â Â Â Â Â Â continue
> + Â Â Â Â Â Â Â Â Â Â else:
> +             print >>mapto, "  None     <- %d..%d" % (b[0], b[-1])
> + Â Â Â Â Â Â Â Â Â Â Â Â continue
> + Â Â Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â Â Â if type(a) == type(0) and len(b) == 1:
> +             print >>mapto, "%6d     <- %d" % (a, b[0])
> + Â Â Â Â Â Â Â Â Â Â Â Â continue
> + Â Â Â Â Â Â Â Â Â Â elif type(a) == type(0) and type(b) == type([]):
> +             print >>mapto, "%6d     <- %d..%d" % (a, b[0], b[-1])
> + Â Â Â Â Â Â Â Â Â Â Â Â continue
> + Â Â Â Â Â Â Â Â Â Â elif type(a) == type([]) and len(a)==2 and len(b)==2:
> + Â Â Â Â Â Â Â Â Â Â Â Â print >>mapto, "%6d..%-6d <- %d..%d" % (a[0], a[1], b[0], b[1])
> + Â Â Â Â Â Â Â Â Â Â Â Â continue
> + Â Â Â Â Â Â Â Â sys.stderr.write("svncutter: Internal error on %s\n" % ((a, b),))
> + Â Â Â Â Â Â Â Â sys.exit(1)
> +
> +def select(source, selection):
> + Â Â "Select a portion of the dump file defined by a revision selection."
> + Â Â emit = 0 in selection
> + Â Â while True:
> + Â Â Â Â stash = source.read_until_next("Revision-number:")
> + Â Â Â Â if emit:
> + Â Â Â Â Â Â sys.stdout.write(stash)
> + Â Â Â Â if not source.has_line_buffered():
> + Â Â Â Â Â Â return
> + Â Â Â Â else:
> + Â Â Â Â Â Â revision = int(source.linebuffer.split()[1])
> + Â Â Â Â Â Â if revision in selection:
> + Â Â Â Â Â Â Â Â sys.stdout.write(source.flush())
> + Â Â Â Â Â Â Â Â emit = True
> + Â Â Â Â Â Â elif revision == selection.upperbound()+1:
> + Â Â Â Â Â Â Â Â return
> + Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â source.flush()
> +
> +def propdel(source, properties, selection):
> + Â Â "Delete unversioned revision properties."
> + Â Â def delhook(propkeys, propdict, revision):
> + Â Â Â Â for propname in properties:
> + Â Â Â Â Â Â if propname in propdict:
> + Â Â Â Â Â Â Â Â del propdict[propname]
> + Â Â Â Â return (propkeys, propdict)
> + Â Â source.apply_property_hook(selection, delhook)
> +
> +def propset(source, properties, selection):
> + Â Â "Set unversioned revision properties."
> + Â Â def sethook(propkeys, propdict, revision):
> + Â Â Â Â for prop in properties:
> + Â Â Â Â Â Â (propname, propval) = prop.split("=")
> + Â Â Â Â Â Â if propname in propdict:
> + Â Â Â Â Â Â Â Â propdict[propname] = propval
> + Â Â Â Â return (propkeys, propdict)
> + Â Â source.apply_property_hook(selection, sethook)
> +
> +def proprename(source, properties, selection):
> + Â Â "Rename unversioned revision properties."
> + Â Â def renamehook(propkeys, propdict, revision):
> + Â Â Â Â for prop in properties:
> + Â Â Â Â Â Â (oldname, newname) = prop.split("->")
> + Â Â Â Â Â Â if oldname in propdict:
> + Â Â Â Â Â Â Â Â propdict[newname] = propdict[oldname]
> + Â Â Â Â Â Â Â Â del propdict[oldname]
> + Â Â Â Â return (propkeys, propdict)
> + Â Â source.apply_property_hook(selection, renamehook)
> +
> +def log(source, selection):
> + Â Â "Extract log entries."
> + Â Â while True:
> + Â Â Â Â source.read_until_next("Revision-number:")
> + Â Â Â Â if not source.has_line_buffered():
> + Â Â Â Â Â Â return
> + Â Â Â Â else:
> + Â Â Â Â Â Â (revision, stash, props) = source.read_revision_header()
> + Â Â Â Â Â Â logentry = props.get("svn:log")
> + Â Â Â Â Â Â if logentry:
> + Â Â Â Â Â Â Â Â print "-" * 72
> + Â Â Â Â Â Â Â Â author = props.get("svn:author", "(no author)")
> + Â Â Â Â Â Â Â Â date = props["svn:date"].split(".")[0]
> + Â Â Â Â Â Â Â Â date = time.strptime(date, "%Y-%m-%dT%H:%M:%S")
> + Â Â Â Â Â Â Â Â date = time.strftime("%Y-%m-%d %H:%M:%S +0000 (%a, %d %b %Y)", date)
> + Â Â Â Â Â Â Â Â print "r%s | %s | %s | %d lines" % (revision,
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â author,
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â date,
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â logentry.count(os.linesep))
> + Â Â Â Â Â Â Â Â sys.stdout.write("\n" + logentry + "\n")
> +
> +def setlog(source, logpatch, selection):
> + Â Â "Mutate log entries."
> + Â Â logpatch = Logfile(file(logpatch), selection)
> + Â Â def loghook(propkeys, propdict, revision):
> + Â Â Â Â if "svn:log" in propkeys and revision in logpatch:
> + Â Â Â Â Â Â (author, date, logentry) = logpatch[revision]
> + Â Â Â Â Â Â if author != propdict.get("svn:author", "(no author)"):
> + Â Â Â Â Â Â Â Â sys.stderr.write("svncutter: author of revision %s doesn't look right, aborting!\n" % revision)
> + Â Â Â Â Â Â Â Â sys.exit(1)
> + Â Â Â Â Â Â propdict["svn:log"] = logentry
> + Â Â Â Â return (propkeys, propdict)
> + Â Â source.apply_property_hook(selection, loghook)
> +
> +if __name__ == '__main__':
> + Â Â try:
> + Â Â Â Â (options, arguments) = getopt.getopt(sys.argv[1:], "ce:fl:m:p:qr:",
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â ["excise", "flagrefs", "revprop=",
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â "logpatch=", "map=",
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â "quiet", "range="])
> + Â Â Â Â selection = SubversionRange("0:HEAD")
> + Â Â Â Â timefuzz = 300 # 5 minute fuzz
> + Â Â Â Â compressmap = False
> + Â Â Â Â excise = None
> + Â Â Â Â revprops = []
> + Â Â Â Â progress = True
> + Â Â Â Â flagrefs = False
> + Â Â Â Â logpatch = None
> + Â Â Â Â mapto = None
> + Â Â Â Â for (switch, val) in options:
> + Â Â Â Â Â Â if switch in ('-c', '--compressmap'):
> + Â Â Â Â Â Â Â Â compressmap = True
> + Â Â Â Â Â Â elif switch in ('-e', '--excise'):
> + Â Â Â Â Â Â Â Â excise = SubversionRange(val)
> + Â Â Â Â Â Â elif switch in ('-f', '--flagrefs'):
> + Â Â Â Â Â Â Â Â flagrefs = True
> + Â Â Â Â Â Â elif switch in ('-l', '--logentries'):
> + Â Â Â Â Â Â Â Â logpatch = val
> + Â Â Â Â Â Â elif switch in ('-m', '--map'):
> + Â Â Â Â Â Â Â Â mapto = open(val, "w")
> + Â Â Â Â Â Â elif switch in ('-p', '--revprop'):
> + Â Â Â Â Â Â Â Â revprops.append(val)
> + Â Â Â Â Â Â elif switch in ('-q', '--quiet'):
> + Â Â Â Â Â Â Â Â progress = False
> + Â Â Â Â Â Â elif switch in ('-r', '--range'):
> + Â Â Â Â Â Â Â Â selection = SubversionRange(val)
> + Â Â Â Â if len(arguments) == 0:
> + Â Â Â Â Â Â sys.stderr.write("Type 'svncutter help' for usage." + os.linesep)
> + Â Â Â Â Â Â sys.exit(1)
> + Â Â Â Â baton = None
> + Â Â Â Â #if arguments[0] != 'help':
> + Â Â Â Â # Â Â if progress:
> + Â Â Â Â # Â Â Â Â baton = Baton(oneliners[arguments[0]], "done")
> + Â Â Â Â # Â Â else:
> + Â Â Â Â # Â Â Â Â baton = None
> + Â Â Â Â if arguments[0] == "squash":
> + Â Â Â Â Â Â squash(DumpfileSource(sys.stdin, baton),
> + Â Â Â Â Â Â Â Â Â timefuzz, mapto, selection, excise, flagrefs, compressmap)
> + Â Â Â Â elif arguments[0] == "propdel":
> + Â Â Â Â Â Â if not revprops:
> + Â Â Â Â Â Â Â Â sys.stderr.write("svncutter: propdel requires one or more --revprop options.\n")
> + Â Â Â Â Â Â if progress:
> + Â Â Â Â Â Â Â Â baton = Baton("", "done")
> + Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â baton = None
> + Â Â Â Â Â Â propdel(DumpfileSource(sys.stdin, baton), revprops, selection)
> + Â Â Â Â elif arguments[0] == "propset":
> + Â Â Â Â Â Â if not revprops:
> + Â Â Â Â Â Â Â Â sys.stderr.write("svncutter: propset requires one or move --revprop options.\n")
> + Â Â Â Â Â Â propset(DumpfileSource(sys.stdin, baton), revprops, selection)
> + Â Â Â Â elif arguments[0] == "proprename":
> + Â Â Â Â Â Â if not revprops:
> + Â Â Â Â Â Â Â Â sys.stderr.write("svncutter: proprename requires one or move --revprop options.\n")
> + Â Â Â Â Â Â propset(DumpfileSource(sys.stdin, baton), revprops, selection)
> + Â Â Â Â elif arguments[0] == "select":
> + Â Â Â Â Â Â select(DumpfileSource(sys.stdin, baton), selection)
> + Â Â Â Â elif arguments[0] == "log":
> + Â Â Â Â Â Â log(DumpfileSource(sys.stdin, baton), selection)
> + Â Â Â Â elif arguments[0] == "setlog":
> + Â Â Â Â Â Â if not logpatch:
> + Â Â Â Â Â Â Â Â sys.stderr.write("svncutter: setlog requires a log entries file.\n")
> + Â Â Â Â Â Â setlog(DumpfileSource(sys.stdin, baton), logpatch, selection)
> + Â Â Â Â elif arguments[0] == "help":
> + Â Â Â Â Â Â if len(arguments) == 1:
> + Â Â Â Â Â Â Â Â sys.stdout.write(__doc__)
> + Â Â Â Â Â Â else:
> + Â Â Â Â Â Â Â Â sys.stdout.write(helpdict.get(arguments[1], arguments[1] + ": no such subcommand.\n"))
> + Â Â Â Â else:
> + Â Â Â Â Â Â sys.stderr.write(('"%s": unknown subcommand\n' % arguments[0])+os.linesep)
> + Â Â Â Â Â Â sys.exit(1)
> + Â Â except KeyboardInterrupt:
> + Â Â Â Â pass
> +
> +# script ends here
>
> ------------------------------------------------------
> http://subversion.tigris.org/ds/viewMessage.do?dsForumId=495&dsMessageId=2405326
>
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=2406520
Received on 2009-10-12 08:01:46 CEST