Does a Interpretation-Engine fit in phobos ?

[default0 via Digitalmars-d]

On Thursday, 30 June 2016 at 15:53:19 UTC, ZombineDev wrote:
> On Thursday, 30 June 2016 at 14:14:41 UTC, Wyatt wrote:
>> On Thursday, 30 June 2016 at 10:36:44 UTC, qznc wrote:
>>> Ok, seriously, it sounds like an awesome feat, but I don't 
>>> think it is necessary to put it into Phobos. First, a dub 
>>> package, please.
>>
>> Agree.  Does Java even have something like that?
>
> I'm afraid you couldn't be more wrong. Both Java and .NET have 
> many provide ways for generating and executing byte code at 
> run-time. Sometimes this the only way to implement something 
> efficiently when runtime reflection is needed. See for example:
> https://msdn.microsoft.com/en-us/library/mt654263.aspx,
> https://msdn.microsoft.com/en-us/library/mt654267.aspx,
> http://openjdk.java.net/groups/compiler/doc/package-overview/index.html,
> http://asm.ow2.org/,
> https://commons.apache.org/proper/commons-bcel/manual.html,
> https://github.com/cglib/cglib,
> http://jboss-javassist.github.io/javassist/,
> https://www.jetbrains.com/mps/index.html,
> https://docs.microsoft.com/en-us/dotnet/core/api/system.reflection.emit and
> https://www.postsharp.net/features
>
> Also almost every dynamic language has some sort of eval 
> function can be used to evaluate arbitrary code at run-time. 
> This has it's security and maintainability challenges, but 
> without any doubt there are situations where this is very 
> helpful.
>
>> That's sort of the exemplar for "hopelessly overdone standard 
>> library".
>
> LOL, I have never heard about a user complaining that a product 
> has too many features, as long as they don't get in the way. 
> Instead users complain when something is **not available**, 
> because that prevents them from getting their job done. What's 
> wrong with having a module in the standard library that you 
> personally won't use, but others will find helpful?
>
> There are many ways in which code-generation can interact with 
> the language runtime. By including it in the standard library, 
> we can ensure that it is thoroughly tested on all supported 
> platforms. Of course the other benefit is that it can be used 
> from other modules in the standard library for implementing 
> various optimizations (e.g. optimizing regex, linear algebra, 
> data base queries, etc.) Such functionality has been a huge 
> success for .NET. E.g. they enabled some advanced LINQ features 
> which are used under the hood of almost every .NET project.
>
>>> Off-topic: Is it possible/feasible/desirable to let dmd use 
>>> dub packages?
>>
>> DMD shouldn't have to download things from the public internet 
>> to do its job.
>
> I don't think you understood the question. The question is how 
> should DMD's code base be structured / modularized. Of course 
> after the DMD is compiled it shouldn't need to use the 
> internet, but that's not the point. The question is if it's a 
> good idea to split the project in small safe-contained reusable 
> packages. For example, that would allow linters to leverage the 
> compiler lexer and parser instead of implementing their own, 
> which often can't handle all language features.
>
> Another huge area is compiler plugins which are quite popular 
> in Rust and .NET
> https://doc.rust-lang.org/book/compiler-plugins.html
> https://github.com/dotnet/roslyn/wiki/Roslyn%20Overview
>
> About compiler plugins in Rust:
>> In a nutshell Rust plugin's let you write normal imperative 
>> run-time type-checked Rust code that is executed at 
>> compile-time on the AST and they let you do absolutely 
>> anything (File/Network I/O, launching threads, ...) including 
>> rewriting the AST. People use it to extend the language "as a 
>> library": implement coroutines, plain-Rust-to-GLSL libraries 
>> that allow you to write shaders in Rust, GPGPU language 
>> extensions, and also to write very powerful libraries: regex 
>> engines, serialization libraries, database libraries that 
>> connect at compile-time to the data-base to validate your SQL 
>> queries and give you compile-time errors if they are 
>> invalid... EDSLs... All in normal, imperative, run-time Rust 
>> code, without shadow worlds (except for the AST API).
> - some guy on reddit
>
> .NET Compiler Platform ("Roslyn"): Analyzers and the Rise of 
> Code-Aware Libraries:
> https://www.youtube.com/watch?v=Ip6wrpYFHhE
>
>>> I guess it would make sense to extract parts of dmd into dub 
>>> packages. As a next step, dmd could use those packages 
>>> instead of duplicating code.
>>
>> Does it?  Which parts?  I'm afraid I don't see the benefit.
>>
>> -Wyatt
>
> LOL x2

My 2 cents on compiler plugins is that as cool and powerful and 
amazing as they sound they put quite the burden on their user. 
Compilers are complicated beasts. In order to write some code 
that correctly uses their API you first have to learn quite a lot 
of stuff (how to use their specific API, how their AST looks, 
what you can and can't do, why this particular error pops up 
etc). And from what I know in those languages, unlike in D, there 
is no **simple** go-to way of doing work at CT that you just need 
done.

I had the pleasure of writing a pseudo compiler extension for 
static interfaces using Roslyn, working out edge cases with 
generics and stuff was.... interesting. Lots of things to look up 
regarding symbol equality, how to compare what, what to cast what 
into to get at your specific information. I imagine Rust will be 
a similar story.
So while they are insanely powerful and nice, they aren't 
something you want to use often. They are sort of the thing you 
resort to as a last or second-to-last ditch effort at getting an 
idea implemented.

That said, having such a powerful, big library for code analysis 
at your disposal is amazing for tooling. My point is, it is in no 
way a replacement or even a compensation for the awesomeness that 
are D's **simple** mixins and template metaprogramming.