D extensions to python, inline in an ipython/jupyter notebook

John Colvin via Digitalmars-d-announce digitalmars-d-announce at puremagic.com
Thu Jul 2 13:45:47 PDT 2015


On Thursday, 2 July 2015 at 19:51:19 UTC, Laeeth Isharc wrote:
> What is the benefit from using distutils for working with D in 
> a notebook?  There are two standards - the Python one, and the 
> D one.  The advantage of using dub is that it becomes 
> wonderfully easy to pull in D projects from code.dlang.org and 
> to compile your own work developed under dub.  (And dub itself 
> continues to improve).  Linking to a D project of decent 
> complexity via distutils is not my idea of fun.

This was my train of thought too.

> I do agree that for pyd itself it would be nice to retain the 
> option of distutils (although I would love to see dub added 
> also),

pyd does have rudimentary dub support, but it's only for 
embedding python in D, not the other way around. Extension 
support is on my TODO list, it's not complicated.

> One should look at this as an alpha, extremely promising 
> project.
>  It is not any rough edges that are important at this stage, 
> but that it has been done at all (in a form that is already 
> very valuable).
>
> The set of people that want to use computers to explore larger 
> data sets than lately considered comfortable in an iterative 
> manner is not small, even confining It just to finance.  It's 
> of real value to be able to hook in to a proper back end that 
> manages market and static data, with also any data processing 
> and analytics also in D, and then to have a pretty and friendly 
> front end that can just talk to this code on the back with 
> minimal messing around to get there.  It's also very nice to 
> have both Jupyter and an Excel spreadsheet at windows onto the 
> data on your local machine or in the enterprise cloud.
>
> The frictions to starting to play with D in a notebook are much 
> lower than going via the command line, so I really am not sure 
> if we should be worried about making it marketable at this 
> stage rather than useful for doing real work.  It's potentially 
> a nice debugging tool where you are trying to make sense of 
> things that don't tidily fit in a debugging window, and where 
> there is just too much stuff to do it via writefln or logging 
> without putting lots of effort into the infrastructure to find 
> events first.
>
> I do like the simpler syntax (than PyD).  One question is 
> whether you need to wrap every member of a struct.

@pdefRecursive!() or @pdef!(Recursive.yes) or @pdef!(Recursive) ? 
It's totally doable.

> But it's a very useful start as it stands.
>
> As an addendum: not many people prefer to do non system type 
> things in C than a higher level language when time is money.  
> So the mindset is you build a C extension to address the worst 
> bits where Python's true colours shine through (ie when you are 
> in a tight loop within Python interpreter and not spending most 
> of the time in its C libraries.  Or writing C glue to connect 
> Python to another library.
>
> In my view, D is different.  I would rather write in D than 
> Python, and to me it's much better Tor doing serious work.  
> Even for parsing a CSV I prefer it (although one could debate 
> the point).  In any case D is not especially slower to write in 
> than Python (particularly when you include time spent getting 
> the bugs out), and in a decent number of cases it may be more 
> productive.
>
> So why bother with Python at all ?  Better ecosystem for 
> charting and exploring data, and an interpreter is better 
> suited generally for certain kinds of tasks.  Also in some 
> areas more libraries, which can be helpful to get a quick 
> result, so that one can go back and do it properly later.

Libraries, libraries, libraries. In a perfect world we can 
reimplement everything in D and it'll be awesome, but in the 
"getting shit done" world that normal people inhabit they need it 
to work, today. Bridges to other languages gives us that. E.g. I 
have a 3D plotting library in D (still in stealth mode on that 
one) that uses matplotlib via pyd to generate tick labels using 
either the builtin TeX parser & renderer (OK, fast) or actually 
calling out to latex (perfect, slow). Could I have done that all 
myself in D? Sure. Would I? No, i would still have terribly 
rendered tick labels with no TeX support. I don't have time to 
write a TeX implementation, even one as rudimentary and limited 
as matplotlib's.

In short, you're right.

> It would be v helpful to have a Datetime conversion from D.  
> Looks like there is a macro for converting from ymd in 
> datetime.h, so I guess one could just write some code against 
> this API in C, D, or Cython and link it in with D so one can 
> transfer data structures over more easily.

I know just enough about that topic to be very scared.


More information about the Digitalmars-d-announce mailing list