According to official release notes for Python 2.5, "ctypes has been
around for a while, but people still write and distribution hand-coded
extension modules because you can't rely on ctypes being present.
Perhaps developers will begin to write Python wrappers atop a library
accessed through ctypes instead of extension modules, now that ctypes
is included with core Python."
Following this advice, I looked at ctypes, and have found that it is a
compelling replacement for SWIG. I conducted this research in part
because I am mentoring a summer of code student this summer, and I
would like to setup a nice framework which will make it easy for the
student to succeed.
Why is ctypes so great? Well, like SWIG, it ships with a code
generator, which automatically generates bindings for Subversion. But
there's a big difference: The ctypes code generator is, well, a hell
of a lot better than the SWIG code generator -- in fact, it's so good,
that I've already generated 'complete' low-level bindings for
Subversion in Python, which are now available at
http://csvn.googlecode.com/svn/trunk/. (NOTE: These bindings are still
in very early-stage development. They work on my machine, but I
haven't tested them elsewhere yet.)
What does ctypes do that SWIG doesn't? Well, it's a long list, and
there's a lot of technical details involved, but the upshot of it all
is that ctypes generates complete low-level bindings without requiring
any major work, whereas SWIG does not. In other words, if we pick
ctypes over SWIG, we don't need to spend much time focusing on fixing
little bugs in the low-level bindings anymore -- we can start focusing
on writing some high-level bindings which make it really easy for
newbies to use Subversion.
If you're familiar with our Python SWIG bindings, you'll know that our
low-level bindings are, unfortunately, full of bugs. If you try to
write a simple Python program which uses the RA layer, you'll find,
unfortunately, that your program may crash with little explanation as
to why. As a developer who loves working with Python, I find these
unexplained crashes very frustrating.
These crashes in our SWIG bindings are usually caused by bugs in our
SWIG typemaps, so they shouldn't be blamed on SWIG itself.
Fortunately, if we switch from SWIG to ctypes, we won't have to spend
any more time writing or debugging typemaps anymore!
Why use ctypes? Well, ctypes generates low-level bindings which work
great out-of-the-box. You don't need to write any typemaps. The ctypes
bindings just work, straight out of the box!
The SWIG bindings, on the other hand, don't work automatically. To
make it possible for SWIG users to write their own callbacks which can
be called from C, we need to write "thunk" functions in C. We haven't
completed this task yet, nor are we even close to doing so.
To summarize:
- SWIG low-level bindings = incomplete, full of bugs, require lots
of maintenance
- ctypes low-level bindings = complete, work great, require little
to no maintenance
Right now, I'm working on some high-level bindings for Subversion in
Python, and I'd rather base these bindings on low-level bindings which
are complete, and work great. So, I chose ctypes for my project.
Should Subversion choose ctypes, too? I think so.
What doesn't ctypes do?
-- It doesn't generate high-level bindings for Subversion. Like SWIG,
it only generates low-level bindings.
-- It doesn't modify the C API in any way. It simply generates API
functions in Python which work exactly the same as their C equivalents
and have exactly the same interface.
Note that, currently, the API for our low-level SWIG/Python bindings
is noticeably different from our C API. See
http://svn.collab.net/repos/svn/trunk/subversion/bindings/swig/python/README
for details. When we switch to ctypes, these differences in the
low-level bindings will go away, unless we choose to implement them
again in ctypes.
So far, I've only implemented one change to the Subversion C API in csvn:
- All functions returning an error through a svn_error_t
automatically raise a SubversionException() instead.
I don't think we should implement any other changes to the low-level
Subversion C API in csvn. Any code which automatically changes the C
API is likely to introduce bugs unless we are very careful. I would
prefer to keep the low-level API simple (and largely identical to our
Subversion C API), so that we can avoid introducing bugs. For more
details on the rationale behind this, see
http://svn.haxx.se/dev/archive-2007-04/0389.shtml
On top of this simple, low-level API, which is identical to our C API,
I plan to write a set of high-level classes. These classes will make
it easy to use Subversion to perform common tasks, such as adding and
removing files from a repository.
Here's an example code snippet, which atomically replaces the "abc"
branch at "http://svn.collab.net/repos/svn/branches/abc" with a brand
new copy of trunk. This code snippet is useful for folks who want to
rebranch in a single atomic commit.
client = Client("http://svn.collab.net/repos/svn")
txn = client.session().txn()
client.rm("branches/abc")
client.cp("trunk", "branches/abc")
txn.commit("Rebranch the abc branch from the latest revision of trunk")
The above task is significantly easier to implement using csvn
high-level API than it is using the low-level C API. To implement the
above functionality using the low-level C API, you would need to drive
the commit editor by hand, much like mucc does, and this is not
simple.
(Note: The above code sample doesn't work yet, because I haven't
implemented the "cp" function yet in the high-level bindings. Still,
"rm", "put", and "mkdir" are already implemented and work great. See
http://csvn.googlecode.com/svn/trunk/example2.py for a working code
sample.)
Here's a summary of my proposal:
1. Replace the low-level SWIG bindings with low-level ctypes
bindings. Keep the low-level bindings as simple and as similar to the
C API as possible so that they don't need much maintenance or
documentation.
2. Write some really nice and easy to use high-level bindings in
Python, so that end users don't need to use the low-level C API. Write
detailed, friendly documentation for this new API.
Some questions for the list:
- Should we choose ctypes over SWIG?
- Once high-level bindings for csvn are ready, should we deprecate
the buggy SWIG bindings, and recommend that users pick csvn instead?
Cheers,
David
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Tue Apr 10 00:46:37 2007