D Compiler as a Library

Mon Apr 16 05:16:56 PDT 2012

On Monday, 16 April 2012 at 02:34:29 UTC, Ary Manzana wrote:
> On 4/13/12 9:10 PM, deadalnix wrote:
>> Le 13/04/2012 11:58, Ary Manzana a écrit :
>>> Having a D compiler available as a library will (at least) 
>>> give these
>>> benefits:
>>>
>>> 1. Can be used by an IDE: D is statically typed and so an IDE 
>>> can
>>> benefit a lot from this. The features Descent had, as far as 
>>> I remember,
>>> were:
>>> 1.1. Outline
>>> 1.2. Autocompletion
>>> 1.3. Type Hierarchy
>>> 1.4. Syntax and semantic errors, showing not only the line 
>>> number but
>>> also column numbers if it makes sense
>>> 1.5. Automatic import inclusion (say, typing writefln and 
>>> getting a list
>>> of modules that provide that symbol)
>>> 1.6. Compile-time view: replace auto with the inferred type, 
>>> insert
>>> mixins into scope, rewrite operator overloads and other 
>>> lowerings (but
>>> I'm not sure this point is really useful)
>>> 1.7. Determine, given a set of versions and flags, which 
>>> branches of
>>> static ifs are used/unused
>>> 1.8. Open declaration
>>> 1.9. Show implementations (of an interface, of interface's 
>>> method or,
>>> abstract methods, or method overrides).
>>> 1.10. Propose to override a method (you type some letters and 
>>> then hit
>>> some key combination and get a list of methods to override)
>>> 1.11. Get the code of a template when instantiated.
>>> 2. Can be used to build better doc generators: one that shows 
>>> known
>>> subclasses or interface implementation, shows inherited 
>>> methods, type
>>> hierarchy.
>>> 3. Can be used for lints and other such tools.
>>>
>>> As you can see, a simple lexer/parser built into an IDE, doc 
>>> generator
>>> or lint will just give basic features but will never achieve 
>>> something
>>> exceptionally good if it lacks the full semantic knowledge of 
>>> the code.
>>>
>>> I'll write a list of things I'd like this compiler-as-library 
>>> to have,
>>> but please help me make it bigger :-)
>>>
>>> * Don't use global variables (DMD is just thought to be run 
>>> once, so
>>> when used as a library it can just be used, well, once)
>>> * Provide a lexer which gives line numbers and column numbers
>>> (beginning, end)
>>> * Provide a parser with the same features
>>> * The semantic phase should not discard any information found 
>>> while
>>> parsing. For example when DMD resolves a type it recursively 
>>> resolves
>>> aliasing and keeps the last one. An example:
>>>
>>> alias int foo;
>>> alias foo* bar;
>>>
>>> bar something() { ... }
>>>
>>> It would be nice if "bar", after semantic analysis is done, 
>>> carries the
>>> information that bar is "foo*" and that "foo" is "int". Also 
>>> that
>>> something's return type is "bar", not "int*".
>>> * Provide errors and warnings that have line numbers as well 
>>> as column
>>> numbers.
>>> * Allow to parse the top-level definitions of a module. Whit 
>>> this I mean
>>> skipping function bodies. At least Descent first built a the 
>>> outline of
>>> the whole project by doing this. This mode should also allow 
>>> specifying
>>> a location as a target, and if that location falls inside a 
>>> function
>>> body then it's contents are returned (useful when editing a 
>>> file, so you
>>> can get the outline as well as semantic info of the function 
>>> currently
>>> being edited, which will never affect semantic in other parts 
>>> of the
>>> module). This will dramatically speed up the editor.
>>> * Don't stop parsing on errors (I think DMD already does 
>>> this).
>>> * Provide a visitor class. If possible, use visitors to 
>>> implement
>>> semantic analysis. The visitor will make it super easy to 
>>> implement
>>> lints and to generate documentation.
>>
>> SDC have a lot of theses, and I proposed a similar stuff for 
>> its
>> evolution. I think it is easier for SDC than it is for dmd 
>> considering
>> the codebase of both.
>
> Cool! SDC is the way to go. Let's focus our efforts on that 
> project. :-)
>
> One thing I saw in the code is that global variables are used 
> in it (specially in the sdc.aglobal module).
>
> Also, the lexer returns a TokenStream which contains the full 
> array of tokens. This is very slow compared to returning them 
> as they are scanned, when requested. But I guess this is easy 
> to fix as it's hidden behind the TokenStream interface.
>
> I tried to compile a simple file:
>
> void main() {}
>
> > ./bin/sdc main.d
> core.exception.AssertError at sdc.gen.sdcmodule(560): Assertion 
> failure
> ----------------
> 5   sdc                                 0x000000010b87aa2a 
> _d_assertm + 42
> 6   sdc                                 0x000000010b4e2daa void 
> sdc.gen.sdcmodule.__assert(int) + 26
> 7   sdc                                 0x000000010b55d19c void 
> sdc.gen.sdcmodule.Store.addFunction(sdc.gen.sdcfunction.Function) 
> + 92
> 8   sdc                                 0x000000010b55d75b void 
> sdc.gen.sdcmodule.Scope.add(immutable(char)[], 
> sdc.gen.sdcfunction.Function) + 123
> 9   sdc                                 0x000000010b54ee36 void 
> sdc.gen.declaration.declareFunctionDeclaration(sdc.ast.declaration.FunctionDeclaration, 
> sdc.ast.sdcmodule.DeclarationDefinition, 
> sdc.gen.sdcmodule.Module) + 1278
> 10  sdc                                 0x000000010b54e4b2 void 
> sdc.gen.declaration.declareDeclaration(sdc.ast.declaration.Declaration, 
> sdc.ast.sdcmodule.DeclarationDefinition, 
> sdc.gen.sdcmodule.Module) + 146
> 11  sdc                                 0x000000010b54d12e void 
> sdc.gen.base.genDeclarationDefinition(sdc.ast.sdcmodule.DeclarationDefinition, 
> sdc.gen.sdcmodule.Module, ulong) + 374
> 12  sdc                                 0x000000010b54c8d0 void 
> sdc.gen.base.resolveDeclarationDefinitionList(sdc.ast.sdcmodule.DeclarationDefinition[], 
> sdc.gen.sdcmodule.Module, sdc.gen.type.Type) + 564
> 13  sdc                                 0x000000010b54c439 
> sdc.gen.sdcmodule.Module 
> sdc.gen.base.genModule(sdc.ast.sdcmodule.Module, 
> sdc.aglobal.TranslationUnit) + 157
> 14  sdc                                 0x000000010b53432d int 
> sdc.sdc.realmain(immutable(char)[][]) + 3081
> 15  sdc                                 0x000000010b4e283e 
> _Dmain + 54
> 16  sdc                                 0x000000010b87b415 
> extern (C) int rt.dmain2.main(int, char**).void runMain() + 29
> 17  sdc                                 0x000000010b87adc5 
> extern (C) int rt.dmain2.main(int, char**).void tryExec(scope 
> void delegate()) + 45
> 18  sdc                                 0x000000010b87b467 
> extern (C) int rt.dmain2.main(int, char**).void runAll() + 63
> 19  sdc                                 0x000000010b87adc5 
> extern (C) int rt.dmain2.main(int, char**).void tryExec(scope 
> void delegate()) + 45
> 20  sdc                                 0x000000010b87ad48 main 
> + 232
> 21  sdc                                 0x000000010b4e2694 
> start + 52
> 22  ???                                 0x0000000000000002 0x0 
> + 2
> ----------------

https://github.com/bhelyer/SDC/issues/30

Rename main.d to something else and it should work. Gotta fix 
that. >_<