threading issues with D -> C -> Python

[Michael via Digitalmars-d-learn]

On Wednesday, 3 December 2014 at 06:30:07 UTC, Russel Winder via 
Digitalmars-d-learn wrote:
>
> As far as I can tell PyD is still active, but in a non-funded 
> FOSS way,
> i.e. work happens as and when volunteers put time and effort 
> in. I
> haven't tried PyD recently but it worked fine last time I did. 
> If can
> set out what you tried and what didn't work, maybe there is a 
> PyD
> solution, or a fix to PyD to give a solution?

Yeah apparently I might have used the wrong PyD. This might end 
up being all I need to do to fix my problem.


> D's "big problem" is shared objects/dynamic link libraries. 
> Without them
> you cannot interwork with Python at all. I have tried 
> experiments on
> Linux creating shared libraries from D code with C linkage 
> entry points
> to create classic Python extensions, and it appears to work 
> fine. Except
> for having to start up the D heap and thread  management, 
> should they be
> needed. But that is what PyD is there for. If I took my 
> experiments any
> further I would end up recreating PyD or something like it.

This is what I'm doing. I'm using the rt_init() function to setup 
the heap/thread management. Am I missing anything else here? It 
seems the issue is definitely more complicated than that.

I'll also point out that: http://dlang.org/interfaceToC had no 
information on calling rt_init first, although it seems like it 
should.


> It sounds like you are in a similar situation except that you 
> appear to
> have an extra layer of C code. I am not sure a layer of C is 
> needed
> between Python and D, it would be good to know more about why 
> you seem
> to need it.

Well I wanted to compile D code and directly call it in Python, 
and since PyD didn't work for me, I instead tried the python-> C 
interface I already knew 
(https://docs.python.org/2/c-api/index.html) and worked my way 
through a C->D interface, which I understood to be relatively 
simple.

> My guess would be not properly initializing the D 
> infrastructure from
> the incoming Python thread.
>
> I would suggest that you want to avoid threads crossing the 
> boundaries
> and just pass data via a shared channel. Unix pipes seem to 
> work well in
> this context since they provide a language independent data 
> channel.

Yeah I'm leaning in that direction myself, although I might try 
the other PyD library first. I wanted to be able to use the D 
message-passing libraries to do the thread-safety stuff at first, 
because it was much easier than the alternative, but I'm not sure 
that's true anymore.