[SAOC 2023] dfmt rewrite - Weekly update #2

[Prajwal S N]

Hi all,

I've had a busy past week figuring out the next steps in the dfmt 
rewrite:

- The lexer has been completely replaced across the utility 
functions, and dfmt compiles with dmd! Figuring out the Makefile 
changes for the build took some fiddling, but I got it working in 
the end.
- The parser from libdparse has been removed, and dfmt currently 
uses the AST from dmd. At the moment, the AST is successfully 
built, but no transformation passes are available.
- Going forward, I will be working on getting one of the existing 
passes working with the dmd AST. If one works, then the rest can 
be ported pretty easily.
- Tangentially, I realised that dfmt does not perform any 
semantic analysis or use any of the data that the AST provides. 
Validating the correctness of the code being formatted (whether 
it can generate a valid AST or not) is not the responsibility of 
the formatter. So, I'll also be trying some experiments to see if 
we can perform the formatting passes purely using the token 
stream from the lexer. There is a chance we might not need a 
parser at all!