So... let's document dmd

[Joakim via Digitalmars-d]

On Saturday, 16 January 2016 at 15:13:28 UTC, H. S. Teoh wrote:
> On Sat, Jan 16, 2016 at 08:35:40AM +0000, tsbockman via 
> Digitalmars-d wrote:
>> On Saturday, 16 January 2016 at 07:52:11 UTC, Jack Stouffer 
>> wrote:
>> >dmd is now in D; theoretically that should allow for other 
>> >projects to import from it like a normal D project. So why 
>> >not make all of the ddmd modules available from any code that 
>> >is complied with it, i.e. just like Phobos?
>> >
>> >The parser was just an example, there are any number of 
>> >things that one could use from the code base.
>> 
>> That would be cool in theory, but it would likely become a 
>> major drag on development of the compiler itself.
>> 
>> Once you start promoting the compiler's internals as part of 
>> the standard library, you can expect sooner or later that 
>> pressure will mount to bring them up to the same high 
>> standards of API design and stability as Phobos.
>> 
>> While this process would be beneficial to the overall quality 
>> of the DMD code being exposed, it would also turn internal 
>> refactorings into public breaking changes. I really doubt 
>> that's a good trade-off at this point in DMD's development.
>
> I disagree. I think having the dmd itself (lexer, parser, etc.) 
> as a
> library (with the dmd executable merely being the default 
> frontend) will
> do D a lot of good.
>
> For one thing, IDE's will no longer need to reinvent a D parser 
> for the purposes of syntax highlighting; they can reuse the 
> actual parser the compiler itself uses, and thus actually be 
> correct (whereas their own parser may actually only parse a 
> subset of D correctly).  They will also be able to support 
> instant feedback as the user types in D code (highlight syntax 
> errors on the fly, etc.).
>
> This will also help with writing various tools that operate on 
> D code, e.g., pretty-printers like dfmt won't need to roll its 
> own lexer/parser anymore. Many other tools also become possible 
> (automatic refactorings, function call tree analysis, 
> dependency analysis, etc.).
>
> You could even use libdmd in your own D code, thus being able 
> to have
> the equivalent of the `eval` available in many scripting 
> languages
> (except that it will have native performance(!)).
>
> The problem of dmd refactorings becoming public breaking 
> changes is not as big a problem if the public API is properly 
> designed. If the compiler code itself is refactored to 
> interface between its sub-components via cleanly-designed APIs 
> with proper encapsulations, I don't see that it will become 
> prohibitively difficult to minimize public API changes without 
> hampering compiler development efforts. This is the way to go 
> for better compiler code quality anyway, so even if libdmd 
> never happens, the code should *still* be refactored in this 
> way.
>
> Besides, I don't think anyone is sanely suggesting that we 
> allow dmd source files to be free-for-all externally-importable 
> modules. The more likely scenario is that there will be a 
> clearly-defined external interface that is unlikely to change 
> much, whereas the implementation underneath is still free to be 
> changed, refactored, reorganized, rewritten, etc..

While this would all be nice in principle, the reality is that 
dmd is mostly worked on by two people these days 
(https://github.com/D-Programming-Language/dmd/graphs/contributors) and they're unlikely to refactor ddmd to put forth a libdmd.  It will depend on someone caring enough to take it up, as Daniel did with the port of the frontend to D.  As tsbockman said, I don't think we're at the stage where anybody will put in that much effort into this.