Reading IDX Files in D, an introduction to compile time programming

Petar Petar
Sat Aug 22 07:04:19 UTC 2020


On Friday, 21 August 2020 at 20:33:51 UTC, H. S. Teoh wrote:
> On Fri, Aug 21, 2020 at 01:18:30PM -0700, Ali Çehreli via 
> Digitalmars-d-announce wrote: [...]
>> In my case I found a limitation: I cannot "iterate a 
>> directory" and import all file contents in there (the 
>> limitation is related to a C library function not having 
>> source code so it cannot be evaluated).
>
> The actual limitation is that string imports do not allow 
> reading directory contents (the C function can be replaced if 
> such were allowed).  Generally, I don't expect directory 
> traversal to ever be allowed at compile-time, since it opens 
> the door to a huge can o' security worms. :-P
>

I feel like limiting CTFE just gives a false sense of security 
and destroys many interesting use cases. If a part of my build 
system will do directory traversal to build the list of files to 
import, what difference would it make to not have this as a 
single build step. The argument that somehow

     dmd -run gen_code.d | dmd -

Is more secure than just:

     dmd file.d # file.d is allowed to access the FS at CT

makes no sense to me.

See Jai for example. You can run absolutely *any* code at compile 
time. 5 years ago Jai's creator made a demo of running an OpenGL 
game at CT [1]. In the same demo he also used CTFE to validate 
calls to printf. He made the case that while many compilers go 
the route of hard-coding checks for printf style functions in the 
compiler, he thinks that users should be able to implement 
arbitrary checks in their code. And 5 years later, instead of D 
expanding the frontiers of what's possible via CTFE, printf 
checking was hard coded in the compiler [2].

[1]: https://www.youtube.com/watch?v=UTqZNujQOlA
[2]: https://github.com/dlang/dmd/pull/10812/files

I don't need say that unlimited CTFE has been a huge success for 
Jai. What I wish is that we can learn from this stop bringing 
arguments that C people would bring for D's CTFE ("My Makefile 
calls a Python script to generate C code and it's doing just 
fine, so I don't think one should be allowed to run code at 
compile time, as it will make the code just harder to follow").

As another example, types in Zig are first class citizens [3] and 
can be manipulated with CTFE just like any other value. "Type 
functions" in D should just be regular D functions taking types 
as parameters and returning types.

[3]: 
https://ziglang.org/documentation/master/#Introducing-the-Compile-Time-Concept


More information about the Digitalmars-d-announce mailing list