RFC: API Compatibility Concerns

[sorry for the slightly long post]

BACKGROUND:

I took a preliminary patch from plasma (see Issue #890) to change the keywords
handling from a struct to a hash internally, and brought it up to date with the
code (as it stood in May 2004). I provided three patches:

1) replaced the core functionality with the new code, with a compatibility
wrapper, but updated no other internal functions which called the old functions
(thus demonstrating that the API change was completely transparent);

2) updated all internal functions to call the new functions instead of the
compatibility wrappers;

3) added the much asked for feature to have custom keyword support, albeit
through the use of properties as keywords.

Since #3 was controversial, I withdrew that patch, but I still got little or no
support from the core developers for #1 and #2, since they did not provide an
immediate benefit. So the code was not accepted.

THE PRESENT:

Since I am home on medical leave, I have a few days of uninterrupted programming
time to bring the patches up to date, but I discovered that a different patch
(the special file handling) has already added a compatibility layer to one of
the functions which I needed. So, if I were to bring up my patches again, it
would require a second layer of compatibility functions, which is not that big a
deal as long as the project is /open/ to that. I don't recall a formal
discussion of API changes prior to 1.1 (or I would have dealt with this at that
time instead of now, when it is almost too late). I don't have the luxury of
being able to read every single patch which goes into the core, and I simply
missed the changes to svn_subst_copy_and_translate.

THE PROBLEM:

The laudable notion of providing a strong API compatibility guarantees needs to
be matched with an equally strong, conscious decision of what exactly
constitutes the Public API. Just making every function that is used by more
than one set of libraries part of the Public API isn't design; it's accretion.
The function that I am having a problem with, svn_subst_copy_and_translate, is
truly a low level function that requires specific setup before being called. It
is a poor candidate for a public API, since to use it in third party code would
require duplicating much of one of the already existing higher level functions.

THE PROPOSAL:

Before 2.0 comes out, I believe that there needs to be a much stronger
commitment to providing an appropriate Public API, which does not consist of
every single function in all libraries that isn't internal to that library. If
there isn't already a caller/callee analysis program which will map the
codebase, I would even go so far as to offer to write one (in Perl of course ;),
which would document the usage of each function. I have more than a sneaking
suspicion that it would become fairly obvious which functions should be exposed
as part of the Public API and which should be relegated to the level of Private
API (can be used only by the project libraries).

I would even go so far as to argue that the Public API should be made as small
as possible for 2.0 (constituting barely more than would already be covered by
known third-party code). It is easy to add to the Public API in minor releases,
but it is impossible to remove anything except for Major releases. With a well
designed program, there should be a limited number of entry points to prevent
needless duplication of code. This is something that I think that Subversion
2.0 can strive to achieve.

The second part of this is that I believe it should be a mandatory step before
each 1.x release to have a formal discussion of API changes, including (as much
as possible) an examination of outstanding patches which make changes to the
same function signatures. If this had happened before 1.1, I would have made
much more effect to get my patches included. Since we are following this
procedure for updating the API in situ, we should consciously decide that making
a change now may preclude something else which isn't quite ready yet, until the
next major release.

THE QUESTION:

If I were to bring my patches up to date now, would I get grief from people that
adding a third iteration of svn_subst_copy_and_translate would "muddy" the API.
  Frankly, I want to know that now because if that is the case, I'm not
interested in doing the work. I'm already discouraged enough about having lost
the opportunity to get this into 1.1 when it would have been relatively easy.
If the feeling is that people aren't serious about allowing multiple API changes
to a function signature (and it could be that certain functions might go through
4 or more iterations between major releases), then I can't see much use in
proceeding with updating my patches.

This is truly the flip side to the API compatibility guarantee with regards to
adding new functionality. If the project as a whole isn't 100% behind the idea
of having svn_subst_copy_and_translate5 somewhere around 1.6, then the API
compatibility guarantee is an impediment to progress and would auger poorly for
future development, or at least require 2.0 to come out that much sooner, to
accommodate pent up demand for features which can only be awkwardly added to the
existing 1.x framework.

I'm not saying I am expecting people to complain; only that I don't feel like I
have enough free time in my already over committed life to submit patches, just
to have them shot down not on a technical basis but because it makes the API
more complicated. I certainly believe that the odds are very good that almost
every case where a function signature has been changed in the 1.x API are
examples of functions which were not suitable to be part of the Public API in
the first place. But before I commit myself to bring the code up to date, I'd
like to make sure that something isn't going to come along and fire a -1 at me
because I am playing the hand that was dealt me.

Thank you in advance for your comments...

John

-- 
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4720 Boston Way
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5747
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org