My Framework wishlist For D

[Andre Pany]

On Wednesday, 28 April 2021 at 12:47:49 UTC, bioinfornatics wrote:
> Firstly my needs it is around data processing and knowledge 
> extraction so It is no a generalization of the needs. Moreover 
> some tools/frameworks have an alternative in D (often not 
> enough mature)
>
> Data computing:
>  - job scheduling (yarn from hadoop, celery from python or 
> slurm from HPC world)
>  - data storage at least read and write to parquet file 
> (through apache arrow lib)
>  - Multinode processing such it is done by Ray: 
> https://docs.ray.io/en/master/
>  - Data processing «à la» Pandas/Dask
>  - scipy and numpy library
>  - a web project generator such it is done with jhipster: 
> https://www.jhipster.tech/
>  - IA library (maybe), if we can store to parquet that imply we 
> are able to load them from python and run tensorfow, pytorch or 
> other …
>
> and may others things

Regarding reading and writing Parquet files using Apache arrow, 
this is more or less easily possible. You can use DPP, but you 
have some small effort afterwards,  see here

https://github.com/atilaneves/dpp/issues/242

Kind regards
Andre