D extensions to python, inline in an ipython/jupyter notebook

[Laeeth Isharc via Digitalmars-d-announce]

On Friday, 3 July 2015 at 10:52:38 UTC, John Colvin wrote:
> On Friday, 3 July 2015 at 10:23:14 UTC, Laeeth Isharc wrote:
>> On Thursday, 2 July 2015 at 22:55:51 UTC, Laeeth Isharc wrote:
>>>>> It would be v helpful to have a Datetime conversion from D.
>>>>>  Looks like there is a macro for converting from ymd in 
>>>>> datetime.h, so I guess one could just write some code 
>>>>> against this API in C, D, or Cython and link it in with D 
>>>>> so one can transfer data structures over more easily.
>>>>
>>>> I know just enough about that topic to be very scared.
>>>
>>> I should be able to call these from D without changing PyD, I 
>>> think, for D to Python.  Not sure about Python to D, but 
>>> maybe.
>>>
>>> https://github.com/ariovistus/pyd/blob/master/infrastructure/python/python.d
>>
>> Doh!
>>
>> It's already in deimos, but you need to call 
>> PyDateTime_IMPORT(); before calling the conversion function.
>>
>> So all that is needed is the following:
>>
>> module example;
>> import std.datetime;
>> import std.stdio;
>> import pyd.pyd;
>> import std.conv;
>> import deimos.python.datetime;
>>
>>
>> DateTime foo()
>> {
>>     return DateTime(1999,1,1);
>> }
>> extern(C) void PydMain() {
>>     ex_d_to_python((DateTime dt) => 
>> PyDateTime_FromDateAndTime(dt.year,dt.month,dt.day,dt.hour,dt.minute,cast(int)dt.second,0));
>>     def!foo();
>>     module_init();
>>     PyDateTime_IMPORT();
>> }
>>
>> Ideally, shouldn't PyD do this for you ?  I think the code is 
>> already there in make_object.d, but it doesn't work - maybe 
>> because PyDateTime_IMPORT() has not been called if it needs to 
>> be.
>>
>> Thanks.
>>
>>
>> Laeeth.
>
> Aren't there time-zone concerns? Or is this just a mapping 
> between D's std.datetime.DateTime and python's 
> datetime.datetime with tzinfo==None, i.e. a naive date?
>
> Also, there would have to be some serious warning signs about 
> translating from python to D, in that the micro-seconds will be 
> truncated. Doesn't a lack of microseconds make it unusable for 
> tick data?

I just put 0 in the microseconds field as I was too tired to look 
up how D does that for now and it wasn't so important for my 
proof of concept.  But python has microseconds (though not 
smaller, I think).  Numpy and pandas have their own datetime64 
concept - I have not so nice memories of using these, but I am 
sure it must be possible to convert at C API level.  Worst case 
if there are no system calls to do the conversion, just hack up 
the current binary representation of the struct and deal with the 
mess when they change the version.  But I wouldn't suggest you 
put that in PyD unless we can do it via official call!  I do 
think regular datetime.datetime should be in there, though.  I 
think it would probably only need a handful of lines added to the 
file I mentioned - I thought they were already there, but that 
was my own change!

I am not sure where the init date call should go, and I am wary 
of tinkering with PyD when I don't yet know the codebase, nor the 
Python C API.  I think someone who knows it could do it in a few 
minutes.

Is it a problem not having sub micro second resolution?  It might 
be for some people, but then it's a core problem, and they 
probably have the budget to do it themselves and might have their 
own date representation anyway (which nonetheless might most 
conveniently be mapped to datetime64 given all the python 
libraries).

However, it's not my use case.  (I guess those who need are 
scientists and investors who analyze data at a high-frequency 
level, which is not me - 1 minute bars are enough for now, and my 
holding period will be much longer).

It's incrementally a big improvement to start by adding 
datetime.datetime...


Laeeth.