Pandas like features

[mw]

On Friday, 23 October 2020 at 19:31:08 UTC, bioinfornatics wrote:
> As a researcher in BioInformatics I use a lot python numpy 
> pandas and scipy. But I am bored by the slowness of python even 
> with cpython code thanks to the GIL and un-optimized tail 
> recursion.
>
> So I thinks really that D could play a big role in this field 
> with MIR and dcompute.
>
> 1/ what is the state of Magpie which was a GSoC 2019:
>  - Mir Data Analysis and Processing Library
>
> 2/ does the scientific computing field is something that D 
> language want to grow ?
>

I think it's definitely the biggest area and opportunities for D 
to become more popular. GIL, lack of performance, and huge memory 
bloat are such pain in Python.

Probably the best way to move forward is to provide libmir as a 
Numpy/Pandas *drop-in* replacement. (And I've suggested to rename 
Mir as NumD from a marketing / promotional perspective).

For the time being, from the language/lib user's perspective, we 
can just use D/libmir to pre-process the data, and maybe save the 
result as csv/npz for further processing (by ... Python). Build 
or wrap something like tensorflow, I think will need much more 
resource than the D community current have, also I'm not sure if 
it worth the effort.


And from the language perspective, maybe D should adopt 
Python/Numpy's array indexing syntax, specifically:

1) use Python's arr[start:end], in addition to D's arr[start..end]

2) and also allow negative index, instead of [$-1]. (This $ is an 
improvement of Java/C++'s arr[arr.length -1], but still is less 
convenient than Python’s negative index syntax).

Python gained such popularity in scientific computing in the past 
~10 years is not an accident, actually Guido made that happen by 
extending Python's syntax:

https://en.wikipedia.org/wiki/NumPy#History

"""
The Python programming language was not originally designed for 
numerical computing, but attracted the attention of the 
scientific and engineering community early on. In 1995 the 
special interest group (SIG) matrix-sig was founded with the aim 
of defining an array computing package; among its members was 
Python designer and maintainer Guido van Rossum, who extended 
Python's syntax (in particular the indexing syntax) to make array 
computing easier.[6]
"""

Maybe Walter should join one of such SIGs as well :-)