DCT: D compiler as a collection of libraries

[Roman D. Boiko]

On Monday, 14 May 2012 at 16:30:21 UTC, deadalnix wrote:
> Le 14/05/2012 17:00, Roman D. Boiko a écrit :
>> Making it a class would give several benefits:
>> * allow not to worry about allocating a big array of tokens. 
>> E.g., on
>> 64-bit OS the largest module in Phobos (IIRC, the 
>> std.datetime) consumes
>> 13.5MB in an array of almost 500K tokens. It would require 4 
>> times
>> smaller chunk of contiguous memory if it was an array of class 
>> objects,
>> because each would consume only 8 bytes instead of 32.
> Why is this a benefice ?
NNTP error: 400 load at 23.60, try later prevented me from 
answering :)

Because it might be difficult to find a big chunk of available
memory (3.5M vs 14M for this particular case).

>> * allow subclassing, for example, for storing strongly typed 
>> literal
>> values; this flexibility could also facilitate future 
>> extensibility (but
>> it's difficult to predict which kind of extension may be 
>> needed)
>>
>
> I'm pretty sure that D's token will not change that much. If 
> the need isn't identified right know, I'd advocate for YAGNI.
Agree.

>> * there would be no need to copy data from tokens into AST, 
>> passing an
>> object would be enough (again, copy 8 instead of 32 bytes); 
>> the same
>> applies to passing into methods - no need to pass by ref to 
>> minimise
>> overhead
>>
>
> Yes but now you add pressure on the GC and add indirections. 
> I'm not sure it worth it. It seems to me like a premature 
> optimization.
It looks so. Thanks.

>> It would incur some additional memory overhead (at least 8 
>> bytes per
>> token), but that's hardly significant. Also there is 
>> additional price
>> for accessing token members because of indirection, and, 
>> possibly, worse
>> cache friendliness (token instances may be allocated anywhere 
>> in memory,
>> not close to each other).
>>
>> These considerations are mostly about performance. I think 
>> there is also
>> some impact on design, but couldn't find anything significant 
>> (given
>> that currently I see a token as merely a datastructure without
>> associated behavior).
>>
>> Could anybody suggest other pros and cons? Which option would 
>> you choose?
>
> You are over engineering the whole stuff.
I'm trying to solve this and other tradeoffs. I'd like to
simplify but satisfy my design goals.