Help optimizing UnCompress for gzipped files

[Steven Schveighoffer]

On 1/4/18 4:47 AM, Christian Köstlin wrote:
> I added now a c variant, that does always malloc/memcpy/free. Its much
> slower for sure.
> Also I put some output in thats shows when a real realloc happens. Its
> like you said:
> 
> did a real realloc
> did a real realloc
> did a real realloc
> did a real realloc
> did a real realloc
> did a real realloc
> did not a real realloc
> did not a real realloc
> did not a real realloc
> did not a real realloc
> did not a real realloc
> did not a real realloc
> did not a real realloc
> did not a real realloc
> did a real realloc
> did not a real realloc
> 
> funny thing is, i do not always get the same sequence of real reallocs.

Another thing I did was print out the size of the buffer before the 
realloc. In other words:

auto oldsize = buffer.length;
auto oldptr = buffer.ptr;
buffer = (cast(ubyte *)realloc(buffer.ptr, newsize))[0 .. newsize];
if(buffer.ptr != oldptr) writeln("moved: ", oldsize);

Even the sizes malloc chooses for its big leap are not consistent from 
run to run!

I'm wondering if possibly when you keep reallocing a large block, and it 
doesn't have enough memory, needs more from the system, it just tacks it 
onto the end of the existing big block. This would make the performance 
for this microbenchmark highly dependent on not doing any other allocations!

BTW, my numbers are consistent, but there is a difference from yours. 
For example, my C runs are about 430ms, and my fast D runs are about 
650-700ms. I never get the 200ms range runs that you are seeing. I have 
a similar machine, but still running Sierra.

-Steve