Help optimizing UnCompress for gzipped files

[Steven Schveighoffer]

On 1/2/18 8:57 AM, Adam D. Ruppe wrote:
> On Tuesday, 2 January 2018 at 11:22:06 UTC, Stefan Koch wrote:
>> You can make it much faster by using a sliced static array as buffer.
> 
> Only if you want data corruption! It keeps a copy of your pointer 
> internally: https://github.com/dlang/phobos/blob/master/std/zlib.d#L605
> 
> It also will always overallocate new buffers on each call 
> <https://github.com/dlang/phobos/blob/master/std/zlib.d#L602>
> 
> There is no efficient way to use it. The implementation is substandard 
> because the API limits the design.

iopipe handles this quite well. And deals with the buffers properly 
(yes, it is very tricky. You have to ref-count the zstream structure, 
because it keeps internal pointers to *itself* as well!). And no, iopipe 
doesn't use std.zlib, I use the etc.zlib functions (but I poached some 
ideas from std.zlib when writing it).

https://github.com/schveiguy/iopipe/blob/master/source/iopipe/zip.d

I even wrote a json parser for iopipe. But it's far from complete. And 
probably needs updating since I changed some of the iopipe API.

https://github.com/schveiguy/jsoniopipe

Depending on the use case, it might be enough, and should be very fast.

-Steve