Expose the CTFE interpter in Phobos?

[Don]

Ary Manzana wrote:
> On 7/25/11 12:55 PM, bearophile wrote:
>> (This post is related to something I have suggested time ago, to offer 
>> some parts of the D compiler through the D standard library itself (to 
>> use the compiler at run-time for some purposes), as recent versions of 
>> C#-dotnet too do.)
>>
>> There is a part of the D compiler that to me seems more useful than 
>> other ones, I mean the D interpreter used for CTFE. People often add 
>> Lua, Python, MiniD, JavaScript to large C/C++/D programs (I think most 
>> video games contain some kind of interpreter). So is it possible to 
>> offer this part alone to the D programmer? With it you are allowed to 
>> create at runtime small strings of D code and interpret it. No need to 
>> learn and add a second scripting language to do it. Just import a 
>> function execute() from a Phobos module :-)
>>
>> The (hopefully) introduction of some writing function 
>> (http://d.puremagic.com/issues/show_bug.cgi?id=3952  
>> https://github.com/D-Programming-Language/dmd/pull/237 ) will make 
>> run-time D interpretation even more useful.
>>
>> There are disadvantages:
>> - Even with some printing function, CTFE interpreter is limited. Don 
>> will remove some more CTFE limitations like allowing classes and 
>> exceptions, but a "scripting language" that can't read and save files 
>> is less useful; so maybe it can't fully replace a Lua interpreter.
>> - If you want to use this feature you need the whole D compiler at run 
>> time too. The D compiler is probably much bigger than a Lua 
>> interpreter. On the other hand maybe it's possible to push the CTFE 
>> interpreter in a DLL that is normally used by the D compiler, that the 
>> D standard library uses if you want to interpret D code at run time. I 
>> don't know if this DLL is going to be small enough.
>> - CTFE is currently much slower than dynamic/scripting languages like 
>> LuaJIT (that are becoming almost as fast as well compiled D). But in 
>> LDC with the LLVM back-end you have all the tools to create a JIT for 
>> interpted D code too :-) LLVM is not a compiler, it's an aggregate of 
>> parts.
>> - D language is not as simple as a scripting language. In video games 
>> the people that write the 3D engine in C++ are not the same people 
>> that program game logic in Lua. The second group of people has 
>> different skills, and they often are not good programmers able to 
>> write C++ code too. So among other things Lua is used to allow a large 
>> number of people to write how the game has to act, not just hard-core 
>> C++ programmers.
>>
>> Do you know/have use cases for running D code (with current or 
>> near-future CTFE limitations) at run-time?
>>
>> Bye,
>> bearophile
> 
> That's why I suggest abstracting away the CTFE to the backend. DMD's 
> backend could implement it as now, just interpreting things in memory 
> and basically implementing an interpreter from scratch. For LDC the 
> backend could just JIT-compile the functions and execute them without 
> any restriction at all...

There's certainly a lot of potential in that direction, but it's not 
simple. For example:
* Cross compilation. The target CPU is not necessarily the same as the 
one which the compiler is running on. (Even the 64 bit compiler is 
executed in 32 bit mode, so this isn't just a theoretical problem).
* The environment in which the user code runs (eg, user account, 
directory being run from, etc) isn't generally the same as the one which 
runs the build system.