Why CTFE I/O would be awesome: No-Build-Tool Custom Build Steps

[Nick Sabalausky]

"Nick Sabalausky" <a at a.a> wrote in message 
news:jkdho8$1p3a$1 at digitalmars.com...
> "F i L" <witte2008 at gmail.com> wrote in message
> news:sofbqiiyvragxjaxqbix at forum.dlang.org...
>> Kapps wrote:
>>> or even automatically creating bindings for a C header file.
>>
>> I've thought about that before. It would be great to have fully automatic
>> linking to C libs. You could even have a cache system that stores .di
>> files and only regenerates them if the .h file was modified.
>>
>
> That hits on the one irritating limitation of CTFE: No I/O. So that cache
> system would be a no-go right now.
>

(I'm branching this off from "Three Unlikely Successful Features of D")

At least one way I think CTFE I/O could be a really big thing:

A library could include any custom build steps it needed (or at least 
pre-build steps) *in* the library itself without the library's user needing 
to deal with buildscripts/buildtools. You can *kinda* do it now, but it 
requires all the processing be done within the naturally-slow and 
memory-hungry CTFE interpreter (compared to just running already-compiled 
native code), and *everything* will be recomputed on every compile whether 
it needs to be or not. So it'd be more of a curiosity right now than a real 
professional approach. But with CTFE I/O, that can change, and I think that 
could be a very big thing:

For example (and I apologize if this sounds like shameless promotion of my 
project - but it's a good example of the scenarios I have in mind), the 
typical expected workflow in most cases for Goldie Parsing System is like 
this:

1. Run 'goldie-grmc' to compile your grammar definition from the BNF-like 
GRM file to a CGT file ("compiled grammar tables" - ie, precomputed LALR and 
DFA tables). This can take some real non-trivial time for non-trivial 
grammars. 'Couse, I think I can still optimize it more, but the nature of 
LALR still prevents this step from scaling well even when natively compiled, 
let alone if it were to be run inside CTFE.

2. Run 'goldie-staticlang' on the CGT file to generate a D library 
custom-tailered for your grammar.

3. Compile your program (which imports the generated D files from 
'goldie-static'').

Obviously there's improvements I can still make, such allowing 
'goldie-staticlang' to optionally do the job of 'goldie-grmc' (thus merging 
steps 1 and 2). Or, it should be possible to make 'goldie-staticlang' (minus 
any 'goldie-grmc' features) a CTFE-based library instead of an external 
tool, once some DMD bugs are fixed. But there's *still* a custom build step 
required, which you have to manually add into your buildscript, or even run 
manually. And while that build step could theoretically be made CTFE-able, 
it would be much less efficient, and would *considerably* slow down compile 
times for *every* compile.

But:

If CTFE had I/O, I'm pretty sure I could set things up so that *all* you'd 
need to do is step 3: import Goldie and compile your program. Then, inside 
Goldie, there would be some CTFE that checks if the CGT file is missing or 
out-of-date, and if so (and *only* if so), invokes 'goldie-grmc' on it 
automatically - which will run at *full* natively-compiled efficiency.  Then 
it would do the same for 'goldie-staticlang' (although it should be 
reasonable to make 'goldie-staticlang' an CTFE-based library instead of an 
external tool, just with some existing DMD bugs fixed). And finally it would 
import your either-already-existing-or-freshly-rebuilt custom-tailered D 
library. Hell, it could even rebuild 'goldie-grmc' itself if it needed to!

All CTFE needs is system(), file I/O, and date/time stuff, and then need for 
buildsystems could be heavily reduced and make external tools/libs easier to 
use. I can see that being a big selling point for D.

That might even make it possible (though I haven't really thought it 
through) to implement the dependency-searching feature of RDMD as a library.