To help our Google summer of code students get started, I've
implemented some simple higher level Python bindings for
"svn_repos.h". These bindings are by no means complete, but they give
you an idea of what nice higher-level Python bindings might look like.
These modules are, for now, stored at
http://csvn.googlecode.com/svn/trunk/, pending their acceptance into
the main Subversion repository.
Right now, using this module, you can easily commit multiple changes
to a Subversion repository in a single transaction. Here's an example:
# Create a new transaction
txn = repos.txn(author="joecommitter",
message="Create file1.txt and file2.txt")
# You can create a file from a Python string
txn.add("file1.txt", contents="Hello world one!")
# ... or from a Python file
file("/tmp/contents.txt", "w").write("Hello world two!")
txn.add("file2.txt", contents=file("/tmp/contents.txt"))
# You can also commit deletes, moves, and copies, using
# txn.copy and txn.delete. Once you're done all of your
# changes, commit the transaction.
new_rev = txn.commit()
print "Committed revision %d" % new_rev
These Python bindings are written in regular, easy-to-read Python.
Under the hood, they're implemented using ctypes. Check out the
repository implementation at
http://csvn.googlecode.com/svn/trunk/csvn/repos.py
You can also see a more complete example script at:
http://csvn.googlecode.com/svn/trunk/example.py
Some nice features of this implementation:
- With ctypes, we don't need to write any code to support the
low-level Python binding functions. ctypes generates the necessary
code to access low-level functions automatically.
- The low-level bindings are identical to the C interface in every
detail. Functions which return "svn_error_t" in C do the exact same in
Python. Functions which accept pointers to pointers accept the exact
same thing in Python. There are no differences (so there's nothing we
need to explain document, besides pointing to the doxygen
documentation for the C header functions!)
- Using ctypes, you can write callback functions for low-level C
subroutines directly in Python. There's no need to write "thunk"
callbacks. ctypes will even automatically generate a function header
for you which declares the types of your function arguments.
- Using ctypes, you don't need to recompile the bindings to wrap a
new function. You can just ask ctypes for the function you want, and
ctypes will automatically generate the function header for you in less
than a second. (The SWIG bindings, on the other hand, take several
minutes to compile.)
- Using ctypes, there's no need to write typemaps at all.
Some folks, who are used to our SWIG bindings, might miss some
features that are provided by our old bindings:
- Our SWIG bindings convert "svn_error_t" return codes into exceptions
- Our SWIG bindings convert APR arrays and APR hashes into Python
arrays and Python hashes, if you write a whole bunch of boilerplate
typemap code.
- Our SWIG bindings convert some pointer types into output
arguments, if you write the necessary typemap code to do this
conversion.
After working with ctypes for a while, I don't miss any of the
features of our SWIG bindings. In fact, I actually am happy that the
ctypes low-level API is exactly the same as our Subversion C API,
because it is easier for me to understand -- I don't need to think
about typemaps when I read the doxygen documentation.
In my opinion, ctypes is much, much better than SWIG for what we want
to do. So far, I haven't found any problems with ctypes -- I've only
found a few deficiencies with the ctypes code generator, which I'll
explain below.
What's missing in the ctypes code generator?
1) The generator doesn't handle 'define' constants very well.
2) The generator sometimes doesn't handle macros correctly.
3) The generator isn't included by default with Python 2.5, so you
have to download it separately.
4) The generator uses gccxml, which is rather difficult to compile.
To handle problem #1, I wrote a little script to generate the define
constants for us, skipping the ctypes code generator. This script now
lives in the above repository as "generate-defines.py".
csvn/defines.py was generated by this script.
To handle problem #2, you can hand-edit the generated macros to work
as you like. For example, if a macro generated by ctypes references an
undefined symbol "NULL", you can add in a definition for NULL to fix
that problem. ctypes won't overwrite your changes.
To handle problem #3 and #4, I setup our Python bindings so that they
don't depend on the ctypes code generator. We only load in the ctypes
code generator if specifically requested by the developer. Developers
only need the ctypes code generator if they want to add functionality
to the ctypes bindings which is not already present. Further, I also
generated a complete XML definition for Subversion 1.4.3 and uploaded
it to the repository so that developers can still run the ctypes code
generator without installing gccxml.
If you take a look at the repository at
csvn.googlecode.com/svn/trunk/, you should note that csvn/constants.py
and csvn/core.py were autogenerated by the ctypes code generator.
Further, csvn/defines.py was generated by my generate-defines.py
script. If we integrated this code into SVN, these files would likely
be generated during the build process.
Overall, I think that these new ctypes Python bindings will provide a
good start for anyone who is looking to extend the Python bindings. I
won't have a lot of time to work on this project but I hope that we
will have a SoC student this summer who will have time to finish off
these bindings.
Some questions for the list:
- Shall we aim to replace the SWIG bindings with ctypes bindings? Of
course, the old SWIG bindings will need to stay around for
compatibility purposes, but perhaps we should focus new efforts on
improving the new ctypes bindings.
- How good do the new ctypes bindings need to be before we commit
them to trunk alongside the existing swig bindings? Obviously, they
will need to integrate with our build system, and ensure that the
generated code is generated during the build, but are there other
requirements?
Thanks for listening!
David
P.S. If you'd like to read more about the higher-level Python bindings
project, please take a look at my previous mail at
http://svn.haxx.se/dev/archive-2007-03/0788.shtml
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Fri Apr 6 08:21:30 2007