how to use dmd.lexer.Lexer

Richard (Rikki) Andrew Cattermole richard at cattermole.co.nz
Sat Jan 7 00:31:15 UTC 2023


Dmd is a separate code base and is not available in a regular D build.

You have to provide it and then link it in.

The easiest way to do it is by using dmd-fe via dub.

Examples: https://github.com/dlang/dmd/tree/master/compiler/test/dub_package

So something like this:

```json
{
	"name": "lexer_test",
	"dependencies": {
		"dmd:frontend": "~>2.101.0"
	}
}
```

source/lexer_test.d:

```d
module lexer_test.d;

void main()
{
     import dmd.globals;
     import dmd.lexer;
     import dmd.tokens;

     immutable expected = [
         TOK.void_,
         TOK.identifier,
         TOK.leftParenthesis,
         TOK.rightParenthesis,
         TOK.leftCurly,
         TOK.rightCurly
     ];

     immutable sourceCode = "void test() {} // foobar";
     scope lexer = new Lexer("test", sourceCode.ptr, 0, 
sourceCode.length, 0, 0);
     lexer.nextToken;

     TOK[] result;

     do
     {
         result ~= lexer.token.value;
     } while (lexer.nextToken != TOK.endOfFile);

     assert(result == expected);
}
```

And then just ``$ dub run``.


More information about the Digitalmars-d-learn mailing list