struct vs class for a simple token in my d lexer

[Tobias Pankrath]

Quoting your post in another thread:

On Monday, 14 May 2012 at 15:10:25 UTC, Roman D. Boiko wrote:
> Making it a class would give several benefits:

>* allow not to worry about allocating a big array of tokens. 
>E.g., on 64-bit OS the largest module in Phobos (IIRC, the 
>std.datetime) consumes 13.5MB in an array of almost 500K tokens. 
>It would require 4 times smaller chunk of contiguous memory if 
>it was an array of class objects, because each would consume 
>only 8 bytes instead of 32.

You'll still have count the space the tokens claim on the heap. 
So it's
basically the 500k tokens plus 500k references. I'm not sure, why 
you would need such a big array of tokens, though.

Aren't they produced by the lexer to be directly consumed and 
discarded by the parser?

> * allow subclassing, for example, for storing strongly typed 
> literal values; this flexibility could also facilitate future 
> extensibility (but it's difficult to predict which kind of 
> extension may be needed)

If performance matters, why would you subclass and risk a virtual 
method call for something as basic as tokens?


> * there would be no need to copy data from tokens into AST, 
> passing an object would be enough (again, copy 8 instead of 32 
> bytes); the same applies to passing into methods - no need to 
> pass by ref to minimise overhead

I'm using string to store source content in tokens. Because of 
the way string in D works, there is no need for data copies.

> These considerations are mostly about performance. I think 
> there is also some impact on design, but couldn't find anything 
> significant (given that currently I see a token as merely a 
> datastructure without associated behavior).

IMO token are value types.