Help optimizing UnCompress for gzipped files

Steven Schveighoffer schveiguy at yahoo.com
Wed Jan 3 21:33:16 UTC 2018


On 1/3/18 3:28 PM, Steven Schveighoffer wrote:
> 1. The major differentiator between the C and D algorithms is the use of 
> C realloc. This one thing saves the most time. I'm going to update 
> iopipe so you can use it (stand by). I will also be examining how to 
> simulate using realloc when not using C malloc in iopipe. I think it's 
> the copying of data to the new buffer that is causing issues.

Looking at when C realloc actually moves the data, it appears it all of 
a sudden over-allocates very very large blocks, much larger than the GC 
will over-allocate. This is why the GC is losing. After a certain size, 
the GC doesn't allocate blocks to grow into, so we start copying on 
every realloc.

So it's not so much the performance of the copying, but the number of 
times copying is required that is affecting the ultimate performance.

-Steve


More information about the Digitalmars-d-learn mailing list