Idea: Run Time Compilation

[janderson]

Craig Black wrote:
> Since Walter is in the habit of adding super advanced features to D, I 
> thought it possible that this one might eventually make it in.  As always, 
> please let me know if I'm way off base.
> 
> What if we could invoke the compiler from an application at run time and 
> generate a function that would be instantiated on the heap?  The function 
> would be defined by a text string and would be invoked via a function 
> pointer or delegate.  This would allow a script-like capability with 
> fully-optimized performance provided by the D compiler.  I think the D 
> compiler would be ideal for this since it is so fast.  Since the method 
> would be allocated on the heap dynamically, it could also be removed when it 
> is done being used.
> 
> Thoughts?
> 
> -Craig
> 
> 

I like this idea (and have thought about it myself, I haven't brought it 
up yet, because I'm still unsure what the best approach would be).

Michael Ambrash, was doing some something of similar nature for 
Pixomatic (a very fast software renderer that looks like DirectX 7). 
I'm pretty sure his compiler would not map directly to DMD because it 
needed to be the fastest possible however he does point out some 
interesting ideas.   The cool thing is, that its setup to optimise for 
the system it is running on (kinda like java's virtual machine) of which 
(in Ambrash's case) there are thousands of combinations.

For many case, we could change the code for that users environment write 
the algorithm in the most otimal way for the given inputs and change 
dymamic data to constants ect...

System cache can have a dramatic impact on performance which is why 
code-modifcation techniques have been avoided in the past.  Ambrash 
found that one frame was enough to allow the newly generated code to 
propgage into cache.

If you were to encapsulate the code in a function pointer or something I 
guess, you could have 2 stages.  1) compile the function and 2) run it. 
  That way in step 1, DMD could push it into cache.

Here are the articles for anyone who is interested.
http://www.ddj.com/184405765
http://www.ddj.com/184405807
http://www.ddj.com/184405848

Anyhow, we do have DDL.  I'm not exactly sure how the compiler/syntax 
could do a much better job yet (I'm sure its possible).

-Joel