foreach over split string

H. S. Teoh hsteoh at quickfur.ath.cx
Wed Jul 17 16:54:34 PDT 2013


On Thu, Jul 18, 2013 at 01:02:26AM +0200, JS wrote:
[...]
> I can't put together a working example right now(other things to do)
> but the gist of the matter is:
> 
> template strsplit(string n) { enum strsplit = std.string.split(n,
> ","); }
> 
> ... inside a ctfe ...
> 
> foreach(n; strsplit!(s)) // doesn't work
> foreach(n; std.string.split(s, ",")) // works
> 
> To me, this is confusing as hell and "n can't be read at compile
> time" took me a good few hours to realize it was simply using the
> template.  I'm guessing that it has something to do with the enum
> "return type" of strsplit but I've tried other various things.

I think it will help to understand the difference between templates and
CTFE. Even though both are processed at compile-time, they are not the
same thing, and mixing them may not always do what you think it's doing.

Basically, a template is used for generating code, so any template
parameters must be known "at compile-time", which means that it must
ultimately reduce to a typename, or a literal of a built-in type. Note
that this is NOT the same thing as "this variable is being used by CTFE
so the compiler should know what its value is at `compile-time'". At
this conceptual stage of compilation, the compiler hasn't fully
transformed the source code into runnable form yet, so the only thing it
can do at this point is to work with, conceptually-speaking, text-level
entities. It can't handle variables and other such things that only
exist when a program has been transformed into a runnable form and
executed by an interpreter.

CTFE is basically a subsystem of the compiler that has the ability to
run code *after* it has been generated (either via parsing the source
code directly, or via expanding templates). That is to say, the program
is now in a (semi)runnable form, but the compiler hasn't output the
object files / executables yet. Basically kinda like an "early
execution" of your program, if you will. Since this happens in a
conceptually later stage than template expansion, whatever's going on in
CTFE generally can't be fed back into template parameters. Template
expansion must have been completed before CTFE even comes into play.

I think part of the confusion may stem from the liberal use of the term
"compile-time", which may have given a false impression that if entity X
is known "at compile-time", then it should be usable by construct Y
which requires stuff that's known "at compile-time". This oversimplifies
the process of compilation; in reality, "compile-time" consists of
several distinct conceptual phases (in implementation this may not
necessarily be true literally, but it helps to understand the conceptual
stages). What generally happens is that the compiler needs to:

1) Read the text representation of the source code, break that down
into tokens ("lexing"), and based on the sequence of tokens, construct
an abstract syntax tree that represents the structure of the source code
("parsing");

2) Translate the syntax tree into an internal representation (IR) of the
program that more-or-less maps to the machine code that will eventually
be output;

3) Translate the IR into actual instructions the target CPU can
understand, and output that as the object file / executable.

Conceptually-speaking, template expansion comes somewhere between (1)
and (2): it's a mechanism for producing large numbers of very-similar
subtrees of the syntax tree without requiring the programmer to do lots
and lots of typing. CTFE is roughly somewhere between (2) and (3): the
program isn't fully compiled yet, but enough translation has been done
that it is now in a form that can actually be run (by an interpreter).
In order to get to this point, template expansion must have been
completed first, since otherwise your syntax tree isn't complete yet, so
by definition it can't have been translated into a runnable form yet.
But since template expansion must have already taken place, that means
you can't feed CTFE values back into templates -- since otherwise
template expansion couldn't have already taken place!

So, long story short, template parameters must be known at "text-level",
if we can coin a term for that, the compiler must be able to reduce said
parameters into a concrete type name or a literal of a built-in type.
CTFE, on the other hand, works after template expansion has completed
(conceptually speaking), and so is essentially a kind of "early
execution" of your program. CTFE variables and the like are
"interpreter-level", so they can't be passed back into template
parameters that expect "text-level" input.

The actual details of what the compiler does is, of course, quite a bit
more complex than the above (over)simplified description, but generally
speaking, if you stick by the rule that CTFE values can't be fed back
into the templating system, you will experience much less frustration.


T

-- 
There are two ways to write error-free programs; only the third one works.


More information about the Digitalmars-d-learn mailing list