What is the best way to use requests and iopipe on gzipped JSON file

[Steven Schveighoffer]

On 10/13/17 6:18 PM, Steven Schveighoffer wrote:
> On 10/13/17 6:07 PM, Steven Schveighoffer wrote:

>> I reproduced, and it comes down to some sort of bug when size_t.max is 
>> passed to ensureElems.
>>
>> I will find and eradicate it.
>>
> 
> I think I know, the buffered input source is attempting to allocate a 
> size_t.max size buffer to hold the expected new data, and cannot do so 
> (obviously). I need to figure out how to handle this properly. I 
> shouldn't be prematurely extending the buffer to read all that data.
> 
> The while loop does work, I may change ensureElems(size_t.max) to do 
> this. But I'm concerned about accidentally allocating huge buffers. For 
> example ensureElems(1_000_000_000) works, but probably allocates a GB of 
> space in order to "work"!

This is now fixed. https://github.com/schveiguy/iopipe/pull/12

-Steve