> Thanks for posting that. Is that patch ready for commit, or is it
> intended just as a proof of concept / basis for discussions?
>
Definitely not ready for commit, it is the WIP of what I was hacking on
Sunday. I was just getting a feel for the kinds of things involved, and
attached it to give you a sense of what I was talking about. I even have a
FIXME note in there where I *think* there may be a potential use after free
issue. We may be getting away with it because of the delay of the python
garbage collector, but I plan on taking a harder look at that one before
reaching a conclusion.
> The questions to decide on are:
> > 1. Are you generally comfortable with the build changes necessary to
> > (optionally) build Py2 and Py3 bindings at the same time?
>
> Do you mean that a single 'make install' would install two versions of
> libsvn_swig_py.so, one compiled against CPython 2 and one compiled
> against CPython 3?
>
> Or do you mean, having a (say) configure-time knob that controls whether
> bindings would be built against py2 or against py3?
>
> I think the latter would be uncontroversial. As to the former, it would
> involve more moving pieces, so I would be concerned about maintenance
> overhead of the new build system code. (We have few active developers
> nowadays.)
>
>
Right, I mean something like having "--enable-swig-py2" and
"--enable-swig-py3" so that Py2 and Py3 could be enabled/disabled
independently, with the option of both being built in the same build. I'll
admit it is specifically with a mind towards packaging, since many (all?)
distros include bindings for both versions if possible. I knew that might
be a stretch, which is why I asked ;) I'll plan on implementing the
straight forward either-or method first, and putting the dual-compile
option in a separate commit that can be dropped if desired. Oh and I won't
forget about the ctypes implementation either.
One point in favor of doing both. While Py3 is expected to be used for all
new projects, and upstream support of Py2 is dropping in 2020 [1], still
many existing project are still on Py2 (like for instance Trac [2]). So in
practice, I'd venture to say that both would typically need to be built in
any case.
> > 2. Do you want to pick up 'py3c' as a new dep, or implement the handful
> of
> > necessary wrappers?
>
> I don't know the swig-py bindings well enough to answer that. I guess
> it depends on how much a "handful" would be.
>
> Datapoints: We use 73 distinct CPython API symbols (see attached grep
> results). Of these, only three(?) entry points (all of them PyInt_*)
> appear in py3c/compat.h.
>
>
I'll default to not pulling it in, I don't think it is really needed. But
I try to default to resisting the "Not Invented Here" tendency. However,
adding a new dep, especially for so little gain, probably doesn't make
sense IMHO. The converse is that while some of the calls might not be
currently used, later additions might have to add more and more compat
functions, approaching what is provided in py3c.
Note also the PyString_* variants would go to PyStr_* in py3c (or
svn_swig_py_*_string*() like in my WIP patch), as well as the PyFile_*
calls.
> > 3. Is the assumption of utf8 encoding sufficiently reasonable?
>
> In Subversion, we assume that argv and stdout are encoded in APR's
> "system encoding", but basically everything else — including nearly
> every single string parameter to every public API — is required to be in
> UTF-8. svn_cmdline.h contains some of the exceptions.
>
> I'm not sure whether that information answers your question, though. In
> what concrete cases do the bindings have to convert between str and
> bytes? What are the compatibility considerations for user code
> (consumers of the swig-py2 bindings that want to upgrade to py3)?
>
>
It answers it well enough. The implicit assumption of string handling in
py2 is one of the things py3 set out to address. Since currently py2 are
really making assumptions already, going from 2->3 using utf8 should be
reasonable IMHO.
> > My general plan of attack will be:
> > 1. Replace deprecated Py2 swig functions and syntax with Py2/Py3
> > cross-compatible versions or wrappers.
> > 2. Ensure existing full Py2 support works correctly.
> > 3. Update build to build both, Py2 and Py3, and get Py3 working.
> > 4. Look at the remainder of the python usage to ensure Py3 compliance.
>
> At some point we'll want to switch one of the buildbot builders to
> use python3. (See http://subversion.apache.org/buildbot/all)
>
> Could you explain how you — and we — test patches written in step #1,
> given that "update the build to support py3" only comes later, in #3?
> Is it currently possible to build the bindings against py3 (well, to
> _run_ the build against py3 and get compiler errors about missing
> symbols)?
>
>
My main concern on step #1 is the "not breaking existing py2 bindings",
laying the ground work for the py3 build changes. I'll probably send all
the patches at the same time, but if they can stand alone, hopefully it'll
make review easier.
> Thanks for the very detailed email!
>
>
Err, yeah. My '--verbose' can't really be disabled... :P
Troy
1: https://pythonclock.org/
2: https://trac.edgewall.org/ticket/12130
Received on 2017-10-18 05:50:22 CEST