Is it possible to "cache" results of compile-time executions between compiles?

[TheFlyingFiddle via Digitalmars-d-learn]

On Tuesday, 24 January 2017 at 21:36:50 UTC, Profile Anaysis 
wrote:
> On Tuesday, 24 January 2017 at 16:49:03 UTC, TheFlyingFiddle 
> wrote:
>> On Tuesday, 24 January 2017 at 16:41:13 UTC, TheFlyingFiddle 
>> wrote:
>>> Everything turned out soooo much better than expected :)
>> Added bonus is that mixin output can be viewed in the 
>> generated files :D
>
> Could you post your solution?
>
> I suggest we get a real caching module like above that has the 
> extra feature of hashing the mixin strings.
>
> This way the caching mechanism can validate if the mixin 
> strings have changed. Put the hash in a comment in the output 
> file that used to test if the input string has the same hash. 
> If it does, simply use the output file, else, regenerate.
>
> Adds some overhead but keeps things consistent.
>
>
> (Since I'm not sure what Cache!() is, I'm assuming it doesn't 
> do this)

This is the solution I through together:
// module mixin_cache.d
mixin template Cache(alias GenSource, string from, string path)
{
     import core.internal.hash;
     import std.conv : to;

     //Hash to keep track of changes
     enum h = hashOf(from);
     enum p = path ~ h.to!string ~ ".txt";

     //Check if the file exists else suppress errors
     //The -J flag needs to be set on dmd else
     //this always fails
     static if(__traits(compiles, import(p)))
     {
         //Tell the wrapper that we loaded the file p
         //_importing is a magic string
	pragma(msg, "_importing");
	pragma(msg, p);
	mixin(import(p));
     }
     else
     {
         //We don't have a cached file so generate it
	private enum src = GenSource!(from);
	static if(__traits(compiles, () { mixin(src); }))
	{
             //_exporing_start_ tells the wrapper to begin
             //outputing the generated source into file p
	    pragma(msg, "_exported_start_");
	    pragma(msg, p);
	    pragma(msg, src);
	    pragma(msg, "_exported_end_");
	}

	mixin(src);
     }
}

To make this work I wrap dmd in a d script like this:
(ignoring some details as what i've got is not really tested yet)

// dmd_with_mixin_cache.d

void main(string[] args)
{
    auto dmdargs = ... //Fix args etc.
    auto dmd = pipeProcess(dmdargs, Redirect.stderr);

    foreach(line; dmd.stderr.byLine(KeepTerminator.yes))
    {
        if(line.startsWith("_exported_start_")) {
           //Parse file and store source in a file
           //Keep going until _exported_end_
        } else if(line.startsWith("_importing")) {
           //A user imported a file. (don't delete it!)
        } else {
          //Other output from dmd like errors / other pragma(msg, 
...)
        }
    }

    //Files not imported / exported could be stale
    //delete them. Unless we got a compile error from dmd
    //Then don't delete anything.
}

The cache template and the small wrapper that wraps dmd was all 
that were needed.

usage is somthing like this:

template Generate(string source)
{
     string Generate()
     {
        //Do some complex ctfe here
        //can't wait for the new ctfe engine!
        foreach(i; 0 .. 100_000)
        { }
        return source;
     }
}

mixin Cache!("Some interesting DSL or similar", __MODULE__);

//some_filename is not really needed but can be nice when 
browsing the
//mixin code to see where it came from. (approximately anwyays)

This is it. If you want I can post a full solution sometime later 
this week but I want to clean up what I have first.