Help optimizing UnCompress for gzipped files

[Stefan Koch]

On Tuesday, 2 January 2018 at 10:27:11 UTC, Christian Köstlin 
wrote:
> Hi all,
>
> over the holidays, I played around with processing some gzipped 
> json data. First version was implemented in ruby, but took too 
> long, so I tried, dlang. This was already faster, but not 
> really satisfactory fast. Then I wrote another version in java, 
> which was much faster.
>
> After this I analyzed the first step of the process (gunzipping 
> the data from a file to memory), and found out, that dlangs 
> UnCompress is much slower than java, and ruby and plain c.
>
> There was some discussion on the forum a while ago: 
> http://forum.dlang.org/thread/pihxxhjgnveulcdtadvg@forum.dlang.org
>
> The code I used and the numbers I got are here: 
> https://github.com/gizmomogwai/benchmarks/tree/master/gunzip
>
> I used an i7 macbook with os x 10.13.2, ruby 2.5.0 built via 
> rvm, python3 installed by homebrew, builtin clang compiler, 
> ldc-1.7.0-beta1, java 1.8.0_152.
>
> Is there anything I can do to speed up the dlang stuff?
>
> Thanks in advance,
> Christian

Yes indeed. You can make it much faster by using a sliced static 
array as buffer.
I suspect that most of the slowdown is caused by the gc.
As there should be only calls to the gzip library