Why does this mixin fail to compile?

[Jonathan M Davis]

On Tuesday, July 2, 2024 1:23:42 AM MDT ryuukk_ via Digitalmars-d-learn wrote:
> I said it 2 times already, i don't want string concatenation,
> i'll benchmark later, but not right now, right now i'm looking
> for a functioning code without string concatenation

D has two options for mixins, and both of them require complete statements.

Template mixins mix in the contents of a template rather than strings, but
they only allow you to compile in declarations, because that's all that
templates can contain:

https://dlang.org/spec/template-mixin.html

You can control what is compiled into a template mixin via conditional
compilation based on the template arguments (e.g. with static if or static
foreach), but the declarations do need to be complete just like they would
for any template, so it will really only work for you if you just need a set
of declarations and not if you're looking to build a list of statements to
run.

For anything more complex, you'll need string mixins:

https://dlang.org/spec/statement.html#mixin-statement

A string mixin takes any string that's known at compile time and compiles it
as code, but the string must contain a complete statement (or a sequence of
statements). You can provide the mixin a string literal, an enum, a list of
strings which are concatenated together, a function call, or any other
expression so long as it evaluates to a string at compile time.

You can of course choose to not use string concatenation to create such a
string, but unless you can provide a string literal with the exact piece of
code that you need, you're going to need to actually build a string from
pieces one way or another, and that normally means either appending to a
string and/or concatenating strings together.

The only way that I can think of at the moment which would involve building
a string at compile time but which would not involve explicit allocations
would be to use a static array and set the unused characters to whitespace,
since the compiler will let you mix in a static array of characters as a
string, e.g.

mixin(()
      {
          char[1024] buffer = ' ';
          ... build the string by setting the characters in buffer ...
          return buffer;
      }());

However, given that this is CTFE we're talking about here, I'm pretty sure
that this particular solution not only allocates, but it potentially
allocates more than simply appending to a string would. IIRC, CTFE's current
implementation doesn't actually use mutatation. Rather, it creates a new
value every time that you mutate a variable. And if that is indeed the case,
then each time you set a character in the static array, you'd basically be
duping the array. CTFE does not work in a manner that pretty much anyone who
hasn't actively worked on it is going to expect, and the generated code
isn't necessarily anything like what would be generated for runtime code.
So, not many D programmers are going to be very good at guessing what's
going to be most efficient with CTFE.

Regardless, because a string mixin must be given an expression that
evaluates to a string, your options are basically

1. Use string concatenation in an expression that you pass to mixin.

2. Call a function that returns a string (with the internals of that
function doing whatever is required to generate that string and return it)
and pass the return value to mixin.

3. Create an enum using either #1 or #2 and pass that to mixin.

So, if you want to build a statement from pieces instead of providing a
known statement or known list of statements, and you don't want to use
string concatenation, you basically have to write a function which builds
and returns a string in whatever the most efficient fashion is that you can
come up with.

And if you want to measure the performance of a particular function that
builds a string during CTFE, you _will_ need to measure build times rather
than testing it at runtime because of how differently CTFE behaves from
running code at runtime. And honestly, I expect that you would have to be
generating a lot of string mixins to be able to get a consistent, measurable
difference in build times between different implementations, but if you care
that much about build times, you'll need to experiment to see what the most
efficient way to build a string is for your code. It is certainly true
though that CTFE tends to be pretty inefficient, unfortunately. It really is
a shame that "newCTFE" was never completed.

In general, best practice would be to just build the string in whatever the
simplest, most maintainable fashion would be, particularly since it will
have no impact on the runtime performance of the program, and you could
easily spend more time trying to optimize your build times than you would
ever save by trying generate the string more efficiently, but it's your
code.

- Jonathan M Davis