Hello,
I find inconvenient the way vendor branches "work" under SVN. The book
describes quite a few operations that need to be manually done, and some of
them are really annoying:
"""
To perform this upgrade, we checkout a copy of our vendor branch, and
replace the code in the current directory with the new libcomplex 1.1 source
code. We quite literally copy new files on top of existing files, perhaps
exploding the libcomplex 1.1 release tarball atop our existing files and
directories. The goal here is to make our current directory contain only the
libcomplex 1.1 code, and to ensure that all that code is under version
control. Oh, and we want to do this with as little version control history
disturbance as possible.
"""
This is complex to do, and a careful operation. It's quite easy to get
something wrong. I plan on writing a Python script (svnvendor.py) to
automate this process. This is the design document for this script,
I'll await any comments!
***************
ABSTRACT DESIGN
***************
Terminology
-----------
- "Package" or "vendor package": a third-party set of files (usually a
library), which we want to import and version within a SVN repository.
- "Project": a tree of files being developed within the SVN repository,
which might want to make use of a local copy of vendor packages.
- "Import a package": the act of adding the third-party package to the SVN
repository.
- "Copy of a package": a local versioned copy of an imported project within
a project.
Requirements
------------
This program is trying to make very easy to do the following things:
- Import a 3rd-party package within a SVN repository (maybe even starting
from the .tar.gz distribution).
- Update an already imported 3rd-party package with a new version, in a way
that adds to the repository only the deltas of changes with respect to
the previous version (that is, committed over the previous one).
- Copy an imported package within a project (versioned copy)
- Upgrade/downgrade the copy of the imported package within a project,
*keeping* all the local modifications (commits in the copy of the
imported package).
- Implicit access to vendor packages by the means of their name (without
having to specify the full URL pointing to the imported version of the
package).
Implicit URLs
-------------
Personally, I'm fed up with typing all those long URLs on the command line,
and I would like the script to optionally "do the right thing" with
understanding the layout of a repository.
I understand that there is some genericity in SVN proper that needs to be
preserved, but I'd rather this kind of external tools to make things easier
for people using standard layouts of repository (like 99% of them are
doing).
Thus, for instance, I'll try my best to make so users can type
"svnvendor.py use libcomplex --version=1.1", and the script will
automatically know where is the vendor branch for the current
project/repository and go finding libcomplex in there.
Vendor URL
----------
There are two main layouts I want to be able to support:
1) vendor branches are stored within a directory called "vendor" or
"thirdparty" within the project root (at the same level of "trunk",
"branches", "tags"). Example:
project_A/
trunk/
branches/
tags/
vendor/
project_B/
trunk/
branches/
tags/
vendor/
2) vendor branches are stored in a global directory
(/vendor or /thirdparty) so to be shared among all projects in the same
repository.
vendor/
libcomplex/
libfoobar/
project_A/
trunk/
branches/
tags/
project_B/
trunk/
branches/
tags/
For each given project/repository, I'll thus assume that there is one
directory in which all the vendor packages are imported. This directory is
called the "vendor URL". Within the vendor URL, there is one directory for
each imported package (named after the official name of the package).
Within each package directory, there is one main directory called
"current", into which all the versions of a given product are effectively
committed. Then, there is one tag for each imported version (that is, one
directory copied from "current" at the time of the import, and named after
the version that was imported).
So for instance, the layout will be something like:
vendor/
libcomplex/
current/
1.0/
1.0a/
1.1a/
libfoobar/
current/
4.7.5/
4.8/
Storing vendor URL per project/repository
-----------------------------------------
Since it's too hard (and slow) to try and auto-detect each time where the
vendor ULR is (and it is out of question, IMO, asking the user to type it
everytime on the command line), I believe it makes sense to find out once
and use it forever.
There will be a local configuration file (eg. ".svnvendor" in the home
directory of each user) which will record which vendor branch to use for
each project/repository. I was thinking of a very easy syntax like:
{UUID1}=/vendor
{UUID2}/project-foo=/project-foo/thirdparty
[....]
where the left part is a (possibly empty) repository path relative to a
repository identified by its UUID; the right part is the
repository-relative path to the vendor directory. For instance, any
svnvendor.py command executed from within a working copy of (any branch of)
"project-foo" above, will automatically assume that the vendor branches are
located within /project-foo/thirdparty in the same repository.
Users will run "svnvendor.py init" (see below) to initialize this file in
an easier fashion (and with some auto-detection magic to speed up common
cases and standard layouts).
Upgrade/downgrade of copies of packages
---------------------------------------
One of the requirements is that it should be easy to upgrade/downgrade the
copy of a package used in a project, while maintaing local changes (that
is, changes to a package which are specific for the project).
To make this operation possible, after evaluating some alternatives
(including the usage of svnmerge.py, and/or complex parsing of svn log), I
decided that it is just easier to store the information within a property
of the directory containing the copy of the vendor branch. I plan to have a
couple of properties:
- svnvendor-name: stores the name of the library (which is the name of
the directory within the vendor URL).
- svnvendor-version: stores the current version of the library (which is
the name of the directory tag within the package directory within the
vendor URL).
So, for instance, if a directory hsa a property svnvendor-name with value
"libcomplex", and svnvendor-version with value "1.4", it means that the
directory is the local copy of (the equivalent of)
[VENDOR_URL]/libcomplex/1.4. When the user asks to upgrade to 1.6 (after
having imported it), svnvendor.py will run something like:
svn merge [VENDOR_URL]/libcomplex/1.4 [VENDOR_URL]/libcomplex/1.6 .
which should merge all the changes between 1.4 and 1.6, while keeping the
local modifications. I'm not sure how "svn merge" is going to handle
subsequent upgrades/downgrades in the same version, but this is another
problem. We will see if we have to make this merge operation harder.
**********************
COMMAND LINE INTERFACE
**********************
--------------------------------------------------------------------------
svnvendor.py init [VENDOR_URL] [--project=REPO_PATH]
Initialize vendor branch support. This will have to be executed once for
each user (for each project using vendor branch support). This command
will:
- Find out the URL of the vendor directory. If specified as an argument,
it's easy :) Otherwise, if the command is executed within a working copy,
it will try to be deduced by stripping parts of the repository path and
looking for a directory named either "vendor" or "thirdparty" within the
repository.
- Find out the project root. By default, it assumes that the project root
is the directory immediately above the vendor directory in the repository
path. It can be overridden with --project.
- Write/update an entry "{UUID}/PROJECT_PATH=VENDOR_PATH" in the .svnvendor
config file.
--------------------------------------------------------------------------
svnvendor.py ls [--project=URL]
Displays a list of all the vendor programs imported in the current
repository. URL is optional to understand what we're talking about (can
point anywhere within the vendor branches or within a project), otherwise
the information is deduced from the working copy.
--------------------------------------------------------------------------
svnvendor.py import FILE/DIR [--name=NAME] [--version=VER] [--project=URL]
Import a new version of the program inside its vendor branch. FILE/DIR
points to either a package (.tar.gz/.zip) or a directory with the extracted
package. The name of the program is deduced by the filename/dirname or
specified with --name. The version of the program is deduced by the
filename/dirname or specified with --version. The algorithm will be
something like this:
- Checkout the current version of the program from its vendor branch into a
temporary directory
- Remove all the files but keep the .svn directories
- Copy/unpack the new package into the directory
- Add/remove files as appropriate (future: detect renames and whatnot)
- Commit the new version
- Tag the new version
This requires two commits, though. I guess I will also add a way to split
it into separate steps, so that the user can do the manual first commit
after double-checking what the script did, and then execute another command
to do the tagging.
--------------------------------------------------------------------------
svnvendor.py copy NAME [--version=VER]
To be executed within a working copy. Copy the vendor program NAME within
the current directory of the current project with a remote copy->WC copy,
and set the special properties (svnvendor-name=NAME, svnvendor-version=VER)
in the WC.
--------------------------------------------------------------------------
svnvendor.py upgrade NAME [--version=VER]
Upgrade a vendor package used by the current project to the latest imported
version (or to the specified version). As explained above, this will make
good use of the svnvendor properties to understand the correct svn merge
operation to execute.
--------------------------------------------------------------------------
svnvendor.py info NAME [--project=URL]
Produces a list of information about the vendor program NAME (list of all
the available versions and maybe other information).
--
Giovanni Bajo
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Fri May 19 15:35:59 2006