On Sat, Aug 6, 2011 at 15:19, Hyrum K Wright <hyrum.wright_at_wandisco.com> wrote:
> On Sat, Aug 6, 2011 at 6:19 AM, Greg Stein <gstein_at_gmail.com> wrote:
>...
>> From my quick perusal... it seems that you might not be handling
>> thread states and the GIL. Have I missed something?
>
> Certainly not. I've read the relevant documentation regarding the GIL
> and thread states, but am still trying to grok how to apply it to this
> case. Most of the docs and examples are written for calling C from
> Python (see [1]), rather than embedding Python in C. I'm not exactly
> sure how this changes the semantics.
>
> At some point, I'll need to start worrying about thread states and the
> GIL, and while I'm happy to experiment and learn via trial and error,
> if you've some insights, I'd certainly welcome them.
Sure. It has been a long time, but in the late 90's, I did a bunch of
embedded Python (in particular: the COM support to enable writing COM
objects in Python). And the thread state stuff was my design, falling
out from my work to remove the GIL from the Python 1.4 interpreter.
So... if I can sandpaper off the rusty bits of my brain in this area,
then I can help with the embedding.
First off: we can ignore the specific calls to make it work, but
structurally you want something like our *_with_transaction() type of
invocation. For every FS vtable entry, it will use this (say)
svn_fs_py__invoke_python() call. We can then build the appropriate
acquisition of any locks necessary.
There is also an "interpreter state" to be aware of, but I believe you
can use a single interpreter for this work. Separate interpreter
states are important when you have disparate embedded codebases. In
this case, there is just one. But regardless of the number of
interpreter states, you always need to deal with the GIL. It will be
easiest to keep one interpreter state, and then N thread states. All
of this can be managed within the invoke_python() helper.
Oh. One thing to note: you *do* want to create and store your own
interpreter state (rather than rely on a default). You'll need some
kind of context to hold that. You cannot use a default state because
(say) mod_python may be running in the same process, and you want to
keep fs_py separate from that.
Note that you can't really invoke most Python functions until you hold
the GIL. This is why the invoke_python() arrangement is best. The
callback will always run with the GIL acquired. At that point, you can
build your argument tuple and call into the Python code.
Cheers,
-g
Received on 2011-08-07 09:27:02 CEST