Big Data Ecosystem

[jmh530]

On Thursday, 11 July 2019 at 18:12:15 UTC, bachmeier wrote:
> On Tuesday, 9 July 2019 at 21:16:03 UTC, Andre Pany wrote:
>
>> What I currently really miss is the possibility to read/write 
>> Parquet files.
>
> For the record, this *is* something that can be done because 
> there are R packages (like sparklyr) that do it, and that means 
> you can do it from D as well. Now maybe you mean you want an 
> interface written in D, but the functionality is nonetheless 
> easily available to D programs. I've never worked with Parquet 
> files so I can't comment on the details.

In something like two minutes of googling, I found that Apache 
Arrow [1] has C bindings [2] for parquet's C++ read/write 
utilities. I know nothing about Parquet files, but I imagine this 
would be faster than calling the R packages.

[1] https://github.com/apache/arrow
[2] 
https://github.com/apache/arrow/tree/master/c_glib/parquet-glib