DMD's lexer available on code.dlang.org

[Vadim Lopatin via Digitalmars-d-announce]

On Sunday, 4 January 2015 at 13:09:42 UTC, Rikki Cattermole wrote:
> On 5/01/2015 2:07 a.m., Daniel Murphy wrote:
>> I've created a dub package for the D version of DMD's lexer, 
>> generated
>> automatically from the C++ source.
>>
>> github: https://github.com/yebblies/ddmd
>> dub: http://code.dlang.org/packages/ddmd
>>
>> There are a few annoying limitations, such that it uses dmd's 
>> error
>> printing and allocation functions, and requires configuration 
>> through
>> 'global'.
>>
>> Here is an example program that uses the lexer:
>>
>> ======================
>>
>> import std.stdio;
>> import std.file;
>>
>> import ddmd.tokens;
>> import ddmd.lexer;
>>
>> /////////////////////////
>>
>> void main()
>> {
>>    Lexer.initLexer();
>>
>>    string data = "void blah() {} // stuff";
>>    auto l = new Lexer("myfile", data.ptr, 0, data.length, 0, 
>> 0);
>>    l.nextToken();
>>    do
>>    {
>>        printf("token: %s\n", l.token.toChars());
>>    }
>>    while (l.nextToken() != TOKeof);
>> }
>>
>> ======================
>>
>> Prints:
>>
>> token: void
>> token: blah
>> token: (
>> token: )
>> token: {
>> token: }
>
> I saw that. I'm really looking forward to getting my teeth into 
> it and doing some good old refactoring. Although that will be a 
> while because of the auto generated thing.

I have a bit similar project - lexer for D written in D.
Written just based on "Lexical" documentation page.

https://github.com/buggins/ddc

Trying make it fast and to do as few memory allocations as 
possible.