Executing pure D at compile-time

kenny funisher at gmail.com
Thu Feb 8 05:29:57 PST 2007


this approach is actually quite nice.

In the spawned processes, pragma(lib) won't work and neither will the -L-lmysqlclient work either. It'd be cool to have the D program be able to contact mysql and get data directly out of mysql, but it's far too much of a security risk, I think. With no external libraries (just D code), it would make file opening very difficult (although not impossible, calling directly the kernel). If .so files were created (I'm on linux) it could be possible to put the so execution into a chroot of only the cwd. I dunno if there's a solution in windows though.

TANGENT: though if chroot were possible in windows as well, then libs could be imported (only if they're in the cwd or symlinked) because a makefile would have to put the libs there or symlink them, and a makefile has full access to the machine, so rm -rf / is possible in a make file already. a compromised makefile would be required to make D compromise the machine, because import ("/etc/shadow") won't be possible.

Anyway, I think this is a super awesome idea. Could be REALLY AWESOME!


kris wrote:
> Following on from the "Regex Redux thread, it seems to me there's an 
> easy way to execute pure D at compile-time. A few elements are needed:
> 
> 1) the ability to describe a compile-time function call
> 2) the facility to pass arguments to it, and recieve a return value
> 3) a means of identifiying the D code to execute
> 4) a manner in which the pure D is executed
> 5) a mechanism for ensuring the executed code is docile
> 
> What follows is purely an illustration, since there are a number of ways 
> to achieve the same result:
> 
> 1) Making the call. let's assume standard calling syntax is enabled. 
> Perhaps something like this (at the call site):
> 
> #  char[] result = regex ("[0-9]", "abc123");
> 
> That may not be an entirely practical syntax, but it hopefully gets the 
> idea across?
> 
> 
> 2) Argument passing. The D source file is text, so a simple 
> implementation might pass text-arguments to the compile-time function. 
> Given that a mixin() is text-based also, it might make sense for the 
> function to return text too e.g.
> 
> # char[] regex (char[] pattern, char[] string) {}
> 
> Note that this function is composed of nothing but standard D code. The 
> args are represented by strings for the sake of simplicity; but it could 
> also be something more sophisticated.
> 
> 
> 3) Function identity. The compiler would need to distinguish between 
> compile-time functions and all other code (so that it knows what is 
> what). One way to do this is to introduce a variation upon public / 
> private / package, called 'extension':
> 
> # extension char[] regex (char[] pattern, char[] string) {}
> 
> With the 'extension' keyword, the compiler can identify 'regex' as a 
> compile-time function. Thus, the regex call noted earlier would be 
> evaluated as a /compile-time invocation/ of the regex function; as 
> opposed to a runtime call. Note that these 'extension' functions would 
> be omitted from the target binary: they are for compile-time use only.
> 
> 
> 4) Execution. One could write a D interpreter and embed it in the 
> compiler, but that's perhaps a bit impractical. Instead, why not simply 
> recurse the compiler (or spawn a child instance) to generate a seperate 
> binary instance of the compile-time function?
> 
> Under Win32, for example, the binary could be generated as a .exe file, 
> and be passed arguments as normal. The return of the function could be 
> captured via an stdout pipe. Better, a dll could be generated instead, 
> and be dynamically bound to the executing compiler. The latter has 
> several benefits, the most obvious being raw throughput.
> 
> 
> 5) The problem with enabling pure D at compile-time is a catch-22. You 
> want the expressive power and raw execution speed, but you want to 
> ensure it doesn't do anything bad. This is a problem regardless of how 
> #4 is implemented. However, I rather suspect the OS will provide the 
> answer for such concerns? I mean, doesn't Vista (for example) provide an 
> execution 'sandbox' where the target is not permitted to create any 
> handles? Without handles, there's no file, socket or registry access. 
> That's a nice sandbox.
> 
> 
> How about an illustrative example? The regex discussed previously?
> 
> ========
> module main;
> 
> import regex;
> 
> void main()
> {
>     // result is generated at compile-time ...
>     char[] result = regex ("[0-9]", "abc123");
>     writefln (result);
> }
> 
> -----
> 
> module regex;
> 
> import std.regexp;
> 
> extension char[] regex (char[] pattern, char[] string)
> {
>      auto exp = new RegExp (args[0]);
>      return exp.find (args[1]);
> }
> 
> =========
> 
> Note that import operates here exactly as it does today. As does 
> everything else. The distinction is the introduction of an 'extension' 
> keyword, and the mechanism to invoke the described function at 
> compile-time from a call-site (rather than generating a runtime call).
> 
> All told, the various posts on compile-time functionality are really all 
> about compiler extensions. The degree of extension is just different 
> across posts. Supporting a pure D approach is certainly better than 
> inventing another language inside D itself; is it not?
> 
> Taking this a little further, there's no need for the 'extension' code 
> to be generated for each invocation. It can easily be cached by the 
> compiler at runtime; particularly a dll implementation. Indeed, assuming 
> the sandbox is in place, there's nothing to prevent one from using 
> pre-compiled extensions instead:
> 
> ==========
> module regex;
> 
> extension char[] regex (char[] pattern, char[] string);
> ==========
> 
> In this case, the extension is simply /declared/ like an extern D 
> function would normally be. (there's a assumption that the dll name 
> would be somehow bound to the function name. And, of course, the 
> assumption that one can sandbox).
> 
> The beauty of this approach is in the simplicity and the power. Occam's 
> Razor would appear to be at work.
> 
> Thoughts?
> 
> - Kris
> 
> 



More information about the Digitalmars-d mailing list