On Mon, Nov 20, 2006 at 10:38:35PM -0600, Les Mikesell wrote:
[Les] On Mon, 2006-11-20 at 19:28, John Rouillard wrote:
[rouilj] How about cherry pick multiple files at different revisions without
[rouilj] going crazy? See any of my prior emails/use cases on the issue.
[Les]
[Les] If you create the tag from a workspace instead of a revision
[Les] you can tag exactly what you want.
This presupposes that there is a single workspace where everything is
checked out. See my comments about it below.
[Les] But, there is no reason to think that the set of files you want
[Les] to tag ever existed (without concurrent changes you don't want)
[Les] at any repository revision.
[rouilj]
[rouilj] Exactly. There is no repository revision that matches the files I need
[rouilj] to tag.
[rouilj]
[rouilj] E.G. version 4 of file1 and version 6 of file2 and version 9 of file
[rouilj] a/b/file9 will all have the same label. Since a new version of file1
[rouilj] was checked in at version 7 and a new version of file2 was checked in
[rouilj] at version 8 I don't have a single revision that will match that
[rouilj] label.
[rouilj]
[rouilj] So I can set labels to track exactly which files I need to have in a
[rouilj] checkout.
[rouilj]
[Les] That is, there will be a point where
[Les] you commit your last change and have exactly the state you want
[Les] to tag.
[rouilj]
[rouilj] There will never be that point in a loosely coupled system.
[Les]
[Les] What 'couples' it at all then? The scenarios I've pictured
[Les] all involve being able to reproduce some existing set of
[Les] files that have been together. Are you saying you want
[Les] to attach a label to some arbitrary set of files without
[Les] ever assembling them in a workspace?
Sort of. The only workspace where they all must be checked
out/assembled is the one that is used to distribute the files to the
end computers. This is a read only workspace except in critical
conditions so will never be used for a checkin.
Some users (e.g. me) do keep an entire trunk working copy around, but
it is not in a publishable state (in work data is present). I could
just check out the subset of the repo I need and use it for these
changes as some people do in which case I would never have the entire
workspace checked out at all. (I probably should work this way since
I have accidentally checked in some files before their time 8-().
Usually the workflow is more like:
I checkout the ldap sub-tree under the trunk, make a change and
check it back in. Dave reviews my change (from the repository, not
a working copy) decides it's good and promotes it to production. In
CVS this was a couple of commands to move the PRODUCTION label and
place a new permanent label for each reviewed file. It was very
straight forward.
The distribution mechanism runs and updates to get the files that
were promoted to the production tree (in CVS a label/tag on the
trunk). So my change now exists as a small change in the production
system. Of the 400+ files in there, I only changed 1 file and I
never needed (nor did anybody else) to check out all 400+ files.
[rouilj] Each element in the system will be independent and I need to
[rouilj] find out what set of independent files is put in place. I
[rouilj] commit my files that have to be released, but there are 3
[rouilj] other admins with their changes in testing stages that should
[rouilj] not yet be published. So I need a way to label the verified
[rouilj] production files and handle all of the files as a single
[rouilj] unit.
[Les]
[Les] You can commit yours to the trunk where they will accumulate
[Les] a change history and copy to a tag where they are easy to
[Les] find. Who decides that they are verified and that the set is complete?
It should be somebody other than me. The problem is having to copy the
handful of files at a differing revisions to the tag that is a clone
of a prior tag except for these few files. This is just messy and
error prone.
[Les] Copying a workspace to a tag is as natural as it gets if you are
[Les] trying to identify exactly that set of files and particularly if
[Les] you are making the last change and committing it because it may
[Les] be impossible to identify that exact set any other way.
[rouilj]
[rouilj] At a rate of 10+ tags a day that makes managing the filesystem name
[rouilj] space a bit nuts.
[Les]
[Les] Do you expect to run out of sequential numbers?
No, but having a thousand+ tags (say a 3 month history) seems a bit
ridiculous. I suppose I could hack together some serial time format so
that the tags could be easily related to time and allow a fast
rollback. We did something similar for CVS. What I am thinking is:
move old production tag to PRODUCTION_YYYYMMDDHHMMSS
create new PRODUCTION tag using copy to copy the old
production tag.
remove the files to be updated from the PRODUCTION tag
copy the new files into the PRODUCTION tag
use the new tag.
This will be a pain when we restructure the tree. As I see it labels
would move with the original files so file a/b/file that is labeled
PRODUCTION at rev 10 would still be labeled at revision 10 after the
move but be at b/c/file. What I would expect on a checkout using the
PRODUCTION label would be the file b/c/file at revision 10.
To get the earlier structure you would have to use a peg revision.
[Les] Assuming that you don't forget to commit before the copy, what
[Les] subsequent problems are likely?
[rouilj]
[rouilj] It's really ugly and prone to error. I have to:
[rouilj]
[rouilj] find the prior tag that the new changes will be based on
[Les]
[Les] You'd need that information regardless of what it is.
Nope, well I do but it's implicit in the system and requires no effort
on my part. I'd just push a new label for the few files I needed to
change. I have no need to lookup the label for all the other files
since they stay where they are and are still valid.
I would need the prior tag info if I was to try to retrieve the prior
release. Under svn I see it as using a peg revision like:
svn co -r PRODUCTION http://repo/config/trunk/ldap/ldap.conf.solaris@194
or some such to get the prior release. In cvs we date tagged the
PRODUCTION tag location as well so it really got two labels:
PRODUCTION and PRODUCTION_YYMMDDHHMMSS. The date tagged one would
never change. Since you could list all the tags on a file in CVS it
was pretty easy to find the prior version although backtracking was a
pain if there were a lot of files since it required a per file revert,
but a shell script helped with that. I was hoping that change sets
under SVN would help with that and I think it does to an extent.
(as an aside SVN's current copy functionality definitely needs to leave
a marker on every file that it copies saying where it was copied to
and at what revision. A non-versioned revprop would have to be used I
guess since I can copy from anywhere not just the head revision I can't
use versioned properties.)
[rouilj] check out this tag
[rouilj]
[rouilj] svn co http://repo/config/production/release100
[rouilj]
[rouilj] switch the files to the trunk release (which I think is impossible,
[rouilj] I would have to switch the ldap directory)
[rouilj]
[rouilj] svn switch http://repo/config/trunk/ldap/ldap.conf.solaris ldap.conf.solaris
[rouilj] svn switch http://repo/config/trunk/ldap/ldap.conf.linux ldap.conf.linux
[rouilj] svn switch http://repo/config/trunk/ldap/ldap.conf.aix ldap.conf.aix
[rouilj]
[rouilj] update the individual files to the code reviewed versions (and make
[rouilj] sure I don't update other files in the directory by running say
[rouilj] 'svn up')
[Les]
[Les] Is this all arbitrary concerning who will want what files in their
[Les] workspace or a continuing story? I'd expect the next round to
[Les] involve keeping the previous workspace, updating to pick up
[Les] whatever other work is current.
Yup. Rinse later repeat 10+ times a day for different people.
[Les] But if these files
[Les] really don't have anything to do with each other what's the
[Les] point of putting them together and then fighting over which
[Les] version is which?
Because all the files are pushed/pulled to the end systems. So they
are not coupled in the classic sense of an application, but they do
have (implicit) interrelationships when they are actually
deployed. When deployed they are all configuring a box for it's
purpose.
It is this configuration (for many systems) that is the real object I
need to version control. It is made up of small components that can be
individually tested and deployed, but the whole set is what needs to
be released. Think of doing 5-10 (or more) software releases a day.
That said, I can have couplings between files. But it is based on what
is changing in the files.
E.G. for resiliency I have the LDAP server host/ip mappings in the
hosts file. If I change LDAP servers, I have to make changes to the
hosts file to match the change in the LDAP servers file. In this case
the files are coupled and it would be a single change/checkin in SVN
(in theory).
However I can also make changes to the LDAP files that have no
coupling to the hosts files. So the coupling is dynamic and depends on
the change being made.
Changes that are made to provision a DNS server have to occur with the
changes that have systems talking to the new DNS server. So it is
coupled but in a much looser way than say software.
[Les] Why not give them their own branches or at
[Les] least subdirectories, with the last operation by whoever is
[Les] blessing them as ready being to merge or move them to the right
[Les] place so updates always pick up the right versions.
Not sure I follow here. The files do form a tree under the trunk
already. That is how access control and things are checked.
Well merge is kind of overkill. For somebody else to do the merge, it
requires a fresh (potentially mixed revision) checked out copy. In
this case a merge isn't really needed since nothing goes to the
production tree except from the trunk. Its like merging a branch in
where the branch is always at the HEAD of the trunk. The PRODUCTION
tree should/will consist of only (cherry picked) revisions from the
trunk.
[rouilj] svn up ldap.conf.solaris ldap.conf.linux ldap.conf.aix
[rouilj]
[rouilj] create a new tag from the working copy
[rouilj]
[rouilj] svn copy . http://repo/config/production/release101
[rouilj]
[rouilj] check out the new tag into the distribution system
[rouilj]
[rouilj] svn switch http://repo/config/production/release101
[rouilj]
[rouilj] and update
[rouilj]
[rouilj] svn up
[rouilj]
[rouilj] With labels I:
[rouilj]
[rouilj] svn label -r 195 PRODUCTION http://repo/config/trunk/ldap/ldap.conf.solaris
[rouilj] svn label -r 196 PRODUCTION http://repo/config/trunk/ldap/ldap.conf.linux
[rouilj] svn label -r 199 PRODUCTION http://repo/config/trunk/ldap/ldap.conf.aix
[rouilj]
[rouilj] (note no working copy needed). Or maybe:
[Les]
[Les] If multiple people are committing all the time, how do you know
[Les] which rev's you want together?
When the person is implementing X they know which files they worked on
and which revs of those files they tested. With 400+ files in our
small sized config most of the time their change would be the HEAD for
a file. We do have some hot files that change more often where this
wouldn't be the case however. The reviewer (if needed and enforced by
scripts) is responsible for promoting the changes to all the
production systems.
[rouilj] svn label PRODUCTION http://repo/config/trunk/ldap/ldap.conf.solaris@195 \
[rouilj] http://repo/config/trunk/ldap/ldap.conf.linux@196 \
[rouilj] http://repo/config/trunk/ldap/ldap.conf.aix@199
[rouilj]
[rouilj] Now if I have the files in a working copy at the proper revisions I can:
[rouilj]
[rouilj] svn label ldap.conf.solaris ldap.conf.linux ldap.conf.aix
[rouilj]
[rouilj] and then
[rouilj]
[rouilj] svn up -r PRODUCTION http://repo/config/trunk
[rouilj]
[rouilj] for the distribution system. Much easier with a lot fewer steps (2
[rouilj] versus 7) and no working copy to gobble disk space and take minutes to
[rouilj] check out.
[Les]
[Les] Someone has to have a working copy or they can't make changes
[Les] or do any testing. Can't the last person to change it copy
[Les] to the tag?
Not always. Depends on policy. Junior admins working on particular
files, or developers changing their own workstation configurations via
this mechanism may be required to have a "code"/change review and the
reviewer makes the push into production. Some shops that have deployed
this systems require everybody to get a different reviewer except for
some "business as usual" changes that are low impact or well tested
via hook scripts.
E.G. sudo config files going bad is a nightmare. So it is
automatically syntax checked on a pre-commit script so a bad syntax
file can't even get into a repository. However there is no way to
verify that the permissions were properly set to least privilege
except by human review.
So advanced admins may be able to commit and promote (and have their
work scrutinized after the fact in the change email), but junior
admins must have a review before the change is pushed to production.
Sadly I am in the same boat as Tim I think it was who has a lot of
non-programmer/VCS experienced types. Most sysadmins aren't developers
(neither am I for that matter, but at least my degree is in physics)
and the whole copy scheme seems a bit weird. CVS's tags is easier to
explain and for them to understand and makes a cleaner workflow for
the types of things I describe in this thread. Even those who used
other systems like Clearcase/Perforce or were just new to the whole
VCS deal (although a couple had worked under VMS so that may have
colored their VCS model) had no problem with using tags under CVS.
--
-- rouilj
John Rouillard
System Administrator
Renesys Corporation
603-643-9300 x 111
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Tue Nov 21 17:07:28 2006