Making byLine faster: we should be able to delegate this

Steven Schveighoffer via Digitalmars-d digitalmars-d at
Mon Mar 23 18:17:18 PDT 2015

On 3/23/15 7:33 PM, Andrei Alexandrescu wrote:
> On 3/23/15 2:42 PM, Steven Schveighoffer wrote:
>> On 3/23/15 10:59 AM, Andrei Alexandrescu wrote:
>>> On 3/23/15 7:52 AM, Steven Schveighoffer wrote:
>>>> On 3/22/15 3:03 AM, Andrei Alexandrescu wrote:
>>>>> * assumeSafeAppend() was unnecessarily used once per line read. Its
>>>>> removal led to a whopping 35% on top of everything else. I'm not sure
>>>>> what it does, but boy it does takes its sweet time. Maybe someone
>>>>> should
>>>>> look into it.
>>>> That's not expected. assumeSafeAppend should be pretty quick, and
>>>> DEFINITELY should not be a significant percentage of reading lines. I
>>>> will look into it.
>>> Thanks!
>>>> Just to verify, your test application was a simple byline loop?
>>> Yes, the code was that in
>> My investigation seems to suggest that assumeSafeAppend is not using
>> that much time for what it does. The reason for the "35%" is that you
>> are talking 35% of a very small value.
> I don't see the logic here. Unless the value is so small that noise
> margins become significant (it isn't), 35% is large.
>> At that level, and with these
>> numbers of calls, combined with the fact that the calls MUST occur
>> (these are opaque functions), I think we are talking about a non issue
>> here.
> I disagree with this assessment. In this case it takes us from losing to
> winning to Python.

Yes, rethinking, you are right. I was jolted by the 35% thinking it was 
35% of the original problem.

I re-examined and found something interesting -- assumeSafeAppend 
doesn't cache the block, it only uses the cache if it's ALREADY cached.

So a large chunk of that 35% is the runtime looking up that block info 
in the heap. On my machine, this brings the time from .3 down to .2 s.

I also found a bad memory corruption bug you introduced. I'll make some PRs.


More information about the Digitalmars-d mailing list