Help optimizing UnCompress for gzipped files

[Christian Köstlin]

On 03.01.18 22:33, Steven Schveighoffer wrote:
> On 1/3/18 3:28 PM, Steven Schveighoffer wrote:
>> 1. The major differentiator between the C and D algorithms is the use
>> of C realloc. This one thing saves the most time. I'm going to update
>> iopipe so you can use it (stand by). I will also be examining how to
>> simulate using realloc when not using C malloc in iopipe. I think it's
>> the copying of data to the new buffer that is causing issues.
> 
> Looking at when C realloc actually moves the data, it appears it all of
> a sudden over-allocates very very large blocks, much larger than the GC
> will over-allocate. This is why the GC is losing. After a certain size,
> the GC doesn't allocate blocks to grow into, so we start copying on
> every realloc.
That a very intersting finding! If I find some time today, I will also
try a pure malloc/memcpy/free solution in c and also look how many real
allocs are done. Thats funny, because this means, that this whole *2
thing for growing buffers that you learn is well taken care of in libc?
Or does it still mean, that you should allocate in a pattern like this,
for libc's algorithm to kick in? Is there api to see how much realloc
really allocated, or is it only possible by comparing pointers to see if
realloc delivers a new or the old pointer?

--
Christian  Köstlin