Hi Julian,
I appreciate your insight into thinking about the problem and your
knowledge of SVN is evident. You are correct about my intentions, while
it is possible for the user to reuse commands (or scripts) to do such an
operation (local or remote), I was hoping that reuse made the SVN
software development of a capability easier to implement as an SVN
command. I am not against client-side shelving or checkpoints, but I
think that a server-side capability is also useful. I am in favor of
both client-side AND server-side stashing concepts, not necessarily only
doing one or the other. Thus, having a command option like "svn shelve
--remote" will automate the capability making the implementation
consistent across SVN users, repos, projects, versions, operating
systems, etc. My focus is on usability for the person with a lot of
work to do, and for getting a new developer with little or no SVN
experience to learn the capability quickly and feel confident that it's
going to work and not mess up their files when stashing. Also, while
Git stashing is very useful, I think server-side stashing is a variation
on that capability which suits SVN's implementation as a client-server
model. I also really like the idea of using this capability in
conjunction with the changelists, whether the changes are stashed
locally or remotely. And maybe changelists will magically handle
directories in the process of implementing this since we will need to
stash those too <cough> <cough>.
I answered the questions below inline.
Thanks,
Danny
On 11/1/2017 6:36 AM, Julian Foad wrote:
> Hello Daniel. Thanks for your interest in discussing this. Emailing
> this list is indeed the right way to do so.
>
> Having answered that meta question, I am changing the email Subject to
> reflect the substantive topic.
>
> Daniel J. Lacks, PhD wrote:
>> I am interested in the design topic for SavePoints. I was wondering
> > if anyone considered server-side stashes instead of client-side?
>
> I am currently working on Shelving and Checkpointing, features which
> are more or less what the SavePoints page envisions (although not
> necessarily the implementation it describes). References:
>
> https://wiki.apache.org/subversion/SavePoints
> https://issues.apache.org/jira/browse/SVN-3625 "Commit shelving"
> https://issues.apache.org/jira/browse/SVN-3626 "Commit checkpointing"
>> Assuming I understand the shelving request correctly, regardless if
the SavePoints are a new command or not, it looks like they want that
ability to be part of the switch command. I think that is brilliant
from the ease of use perspective.
>> For checkpoints, I have encountered this type of desire myself to
not want to make a commit but needing to backup my work.
>> In both of these cases and stashing/SavePoints capability in
general, in order to achieve these capabilities now, I implement this in
a script to see what files I change and move or back them up somewhere
in my home directory with the same file structure as they exist in SVN.
Sometimes I tar and zip the files up too, in the use case of
checkpointing I suppose. There certainly are manual or scripting
workarounds for making this work, but a capability baked into SVN would
make me feel more confident that I am not goofing something in my script
that will erase my real changes because of a malformed mv command, or
handling a special case like file permission problems (not that either
of these have happened before (ok, maybe they did)).
>
> These are deliberately client-side features for the main reasons of
> speed and offline usability. Speed is critical: one of the main use
> cases is to be able to switch *quickly* between small tasks, and this
> has to be useful for people using a relatively slow WAN connection.
>
> >> I agree with the comments about speed being a concern for stashing,
> but SVN could cache remote stashes locally upon checkouts, switches,
> updates, etc. in the .svn folder and also makes implementation easier
> perhaps for client-side stashing in the process. Thus, if someone
> does a local stash, the files are saved in the .svn directory, but
> perhaps are not sync'ed up to the server unless explicitly requested.
>
>
> Have we thought of the server-side possibility? Of course. (See for
> example the 2nd/3rd comments in those issues.) But Subversion already
> supports saving work to a server-side branch, so we need to ask, what
> do you need for your use case, beyond the present ability to use
> branches? So let's explore further.
>> To answer the use case question (from my perspective off the top of
my head):
1. Must be easy to use. Whether the default is client-side or
server-side or two different commands, make this something easy to pick
up. Also, combining the capability with other commands like merge,
changelists, switches, etc. again makes this easy especially if there is
a way to do it without learning new arguments. Ease of use also comes
into play, in my opinion, by offering the server-side option because now
I don't have to manually sync stuff between servers if I am implementing
changes that go beyond working on one machine. One of my roles is to
train new developers/interns who are junior or still in school in a
professional environment. In most cases they have never used a version
control system ever. They often have to learn many simultaneous things,
usually version control is not the primary learning objective (though I
do think it is super important), so any way to make that process easier
helps.
2. I work a lot on distributed computing concepts, pretty much from
being a student to working as a professional for 16+ years. I typically
work on one branch which may span over any number of computers/VMs. The
changes for one computer/VM (i.e. a web server) may be different than
another (a client) or another (a test driver) which share the common
branch and purpose for a change. So I may need to stash these items,
work on something else, use new VMs with security/OS upgrades and erase
the old VMs, then need to unstash my changes to continue with what I was
working on. There are variations on this theme, and there may be many
computers involved with disparate systems and changes, but I think the
concepts discussed here so far cover this use case. (Note that we have
the ability to create more VMs to work on multiple tasks simultaneously,
but at some point that because onerous to configuration manage, and
consumes shared resources.)
3. An extension of #2, I may not be the only person working on a change
across many computers on the same branch. Considering sometimes I work
with junior engineers, I may code something correctly the first time,
but they may want to try different things to interface with whatever I
am doing. This is where I think the peer review capability mentioned
can really help too so I can see their changes in SVN on the server
without having to SSH into their VMs. Plus if they can stash, then I can
compare approaches while they work on the new approach concurrently (and
not have to wait for me to look at something).
4. While I may want to stash something short term, sometimes things
happen which make it go longer than anticipated. Sometimes this lasts
really long because I move onto a different project then get back to
something a year or more later. Changes on the server, in my
environment, are backed up nightly - local VMs/computers are usually not
backed up. If I encounter VM overload, it would be nice to know with
confidence that I can erase VM to work on other stuff, then recreate
exactly where I left off by recloning a gold image, checking out the
code, then unstashing my previous changes.
5. This brings up another point, the use case assumes I am able to
comment/annotate the changes. I work on simulatenous tasks at any given
time, it is often hard to remember what I am doing and where I left off
which I switch contexts.
6. Something else I thought about is a code handover. If I am working
an issue which is reassigned to a different developer, it would be nice
to stash my code and share it rather than check it in. This is to not
clog the SVN log with unnecessary entries which will only aggravate the
integrators when they need to audit history.
>
>> The stash would work similar to a commit
>> except it would check-in code perhaps in a hidden or protected branch
>> within the svn:stash workspace.
>
> Making namespaces of branches that are 'hidden' or 'protected' is
> something that can potentially be done with server authz rules, but is
> this important for you? Why? If you just designate
> '^[/project]/branches/user/USERNAME/*' as the place for USERNAME's
> branches, with no special protection, does that work?
>> In my opinion, I put this recommendation in the "too complicated"
category. Configuring authz rules is not something the typical user has
access to do in a professional work environment (where an IT or some
type of admin locks down the SVN server). Some IT guys are honestly not
that skilled to go that far as well, sometimes the IT staff may know how
to status a server or reboot the box, that is all. So asking some IT
staff for a capability like this is too complicated. Also, if the
approach is something the users can see in the branches, they are going
to mess it up (i.e. naive people manually editing or not checking in
svn:mergeinfo changes).
>
>> This would allow developers to not only
>> swap workspaces, but to swap them across multiple physical machines or
>> VMs. It is also possible to share those changes with others as well, for
>> example the basic commands to show SavePoints may only show your save
>> points, but perhaps there can be an optional argument to show anyone’s
>> SavePoints either on your branch or any branch. I imagine that swapping
>> to a SavePoint would first work like a switch command to get you to the
>> same point you were (optionally), then a pseudo-merge command to grab
>> the changes and copy them into your local directory. It seems like such
>> a capability may be built reusing some existing functionality.
>
> It certainly can be done by (re)using existing functionality :-)
>
> # save my local changes
> svn copy -m "save" . ^/branches/save/$USER/foo
>
> # revert my local changes, now they're saved as 'foo'
> svn revert -R .
>
> # show my save points
> svn list ^/branches/save/$USER
>
> # show everyone's save points
> svn list ^/branches/save -R
>
> # apply save point 'foo' to my WC
> svn merge ^/branches/save/$USER/foo
>
> There's not even a need for separate 'switch' and 'pseudo-merge'
> steps, if we assume my WC is already on the same branch (e.g. trunk)
> from which the save-point branch 'foo' was created.
>> As I mentioned above, this capability can be scripted either to
cache locally or remotely. But if the stashes appear in the typical SVN
commit history and merge requests, this is probably going to fluster the
integrators and build automation folks, perhaps the systems engineers
that need to create and enforce processes to handle this on each project.
>
> Seriously, though, I can say some things.
>
> I am sure you wish for a user-friendly command-line interface to
> access this scheme, such as "svn shelve --list" (which would translate
> to "svn list ^[/project]/branches/save/$USER" if that is the
> underlying storage scheme).
>> Yes, that and more.
>
> Subversion is intended to be a system which has a core part with
> libraries and a simple command-line interface, that is then extended
> upwards with third-party interfaces such as TortoiseSVN, Cornerstone,
> Visual Studio / IntelliJ / NetBeans IDE integrations, and others.
>
> Maybe this kind of use of branches for 'shelving' is more the job of a
> higher layer of software built on top of Subversion core, or an
> alternative command-line client. Not all features like this should be
> built in to the core. One of Subversion's strengths is the simplicity
> of its command set. Of course this cuts both ways: if this
> functionality is commonly wanted then there should be an easier way to
> access it.
>> This is where I confess I don't know as much about SVN as you, at
this point I am more interested in discussing use cases and high level
design, this may be going a little too low in design. But I am
interested to learn more, so I appreciate the comments. In one regard,
I don't want to jump to any implementation conclusions; but I also don't
want to suggest something that would imply a total re-write or re-design
of what is currently there.
>
> One difficulty here is we (this group of developers subscribed to
> dev@) don't really get involved much in designing Subversion features
> outside the part that we produce ourselves. Maybe we could change
> this. If we draw a diagram showing the core and third-party Subversion
> software, showing what is in the core and what sort of features we
> expect the third-party software to provide, that just might incite
> those third-party developers to go and build those features.
>> That would be useful. I don't know I am the right person to do that
because I am not familiar with all of the tools you mentioned, but I
definitely see the value in integration with other tools. It helps with
the "ease of use" request as well, perhaps if there is a
"Shelve/Unshelve" button or menu option which is easily accessible on
the WC root directory. Also, knowing what to show or not to show when
auditing code, and to consider use cases like a peer reviewer role, a
peer review leader role, a developer role, and an integrator role.
>
> Another angle, touched on by Paul Hammant in his reply, is portability
> of change-sets. Let's say we implement client-side shelving. The next
> logical request is certainly going to be a way to transfer those
> shelved changes easily to a branch from where they can be moved to
> another client machine, shared, backed up, etc. And, in his
> interesting case, transferred to a code review system which is
> separate from the Subversion server.
>
> So "standardized server-side handling of such things", as he puts it.
> What can we do in this direction?
>
> One thing we can do is make a Subversion 'patch' format that is a
> complete serialized representation of any potential Subversion commit.
> The 'svnadmin dump' format is a serialized representation of an actual
> commit, based on a specific previous revision number. For a 'patch'
> representing a potential commit, we don't know the eventual base
> revision yet, and so we need the sort of flexibility in applying it
> that a 'context diff' gives. We need to meld the 'context diff' idea
> (which originally is only defined for plain text) with the ability to
> specify all possible Subversion operations including copies and moves,
> directories, and properties.
>
> Another thing we can do is look at what sort of commands and
> infrastructure will be needed to refer to these change-sets and send
> and receive them. For example, 'shelving' should have 'import' and
> 'export' commands to bring change-sets into and out of the 'shelved'
> space, and also there should be ways to import a change-set directly
> to a new branch and export one from a branch. I say this because if
> you build a system where there are several concepts like 'shelved
> changes' and 'changelists' and 'commits' but for each concept there is
> only exactly one thing you can do with it (for a shelved change: you
> can unshelve into the wc, only) that system becomes limited and
> clumsy. Some things you can do require two or three steps to do, when
> logically they should only require one step.
>> These are all creative ideas, I cannot say I like one more than
another at this point. I recommend fleshing out the use cases (as you
recommended above and below), then perhaps the way forward would be
evident. I appreciate you thinking about the system-level effects of
introducing a change like this and considering how it impacts (good or
bad) other commands and capabilities, that is important for this type of
feature.
>
> So, yes, lots of interest. What do you (all) think?
Thanks again for your response and for listening to the community.
>
> - Julian
---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
Received on 2017-11-03 05:49:22 CET