Request for comments: std.d.lexer

[dennis luehring]

Am 31.01.2013 13:48, schrieb Jacob Carlborg:
> On 2013-01-31 13:35, dennis luehring wrote:
>
>> why not only the symbols at the border needs to be "connected" then
>>
>> the question is: how many blocks(threads) are ok for 1,2,3,8 cores - can
>> also depend on the speed of the filesystem
>
> That would require some profiling to figure out.

i would say it "can" help alot so the design should be able to use 
split-parts and combine symboles at the border

and also file-based lexing should be threadable so that 16 files can be 
handled by my 16 cores system full in parallel :)

but its also size dependend - it makes no sense to split in too small 
parts, could be counter-productive