State of the Compiler

[Walter Bright via Digitalmars-d]

I'm pretty unhappy with the state of dmd. I think the best that can be said for 
it at the moment is that it works.

1. every time we fix something, there are unexpected regressions.
2. the code is a hopeless tangle.
3. too much global state (like global.errors, which is itself a source of 
repeated regressions)
4. it's slowed down substantially.
5. near complete lack of encapsulation
6. my ideas on how software should be written have evolved a lot since it was 
laid out

This cannot be fixed overnight, but I've been working on it off and on in 
between doing other more urgent things, and submitting PRs for them. The goals 
of all this are (and this includes the back end):

1. elimination of all conditional compilation, and macros. All that code should 
be encapsulated and banished to Port or Target.

2. encapsulation of all memory management. For example, StringExp's instance of 
string data is allocated and free'd all over the place. I've made some progress 
in encapsulating it. Getting it properly encapsulated means we can do Small 
String Optimization in it, which should get us a nice boost. SSO for all these 
sorts of structures needs to be done.

3. retrofit in const as much as possible, and later try to use pure.

4. get rid of all global state variables, except for configuration stuff set by 
command line switches

5. convert back end to D.

6. use nested functions to improve encapsulation.

7. remove C++ artifacts, like the fake delegates, and replace them with real 
delegates

8. replace symbol tables and custom hash code with builtin hashes, and fix the 
builtin hashes if they aren't as fast

9. change to "full lazy" semantic analysis of imports, i.e. only parse them far 
enough to build the symbol table. Do semantic analysis of the symbols only on 
demand. I did this for enums a while back, and the results have been very pleasing.

10. use rc memory for CTFE, or even stack memory. this is already done to a 
small extent, but should go much further

11. encapsulate, encapsulate, encapsulate

12. start retrofitting with phobos algorithms

13. use ranges


One benefit of all this is we could start using multicores to compile!

This will all take years, but we've already made good progress such as the 
conversion of the source code to D.