Interpolated strings and SQL

[Nickolay Bukreyev]

Hello. It is fascinating to see string interpolation in D. Let me 
try to spread some light on it; I hope my thoughts will be useful.

1.  First of all, I’d like to notice that in the DIP1027 variant 
of the code we see:

     > `auto fmt = arg[0];`

     (`arg` is undeclared identifier here; I presume `args` was 
meant.) There is a problem: this line is executed at CTFE, but it 
cannot access `args`, which is a runtime parameter of `execi`. 
For this to work, the format string should go to a template 
parameter, and interpolated expressions should go to runtime 
parameters. How can DIP1027 accomplish this?

2.
     > Note that nested istrings are not supported.

     To clarify: “not supported” means one cannot write

     ```
     db.execi(i"SELECT field FROM items WHERE server = 
$(i"europe$(number)")");
     ```

     Instead, you have to be more explicit about what you want the 
inner string to become. This is legal:

     ```
     db.execi(i"SELECT field FROM items WHERE server = 
$(i"europe$(number)".text)");
     ```

     However, it is not hard to adjust `execi` so that it fully 
supports nested istrings:

     ```d
     struct Span {
         size_t i, j;
         bool topLevel;
     }

     enum segregatedInterpolations(Args...) = {
         Span[ ] result;
         size_t processedTill;
         size_t depth;
         static foreach (i, T; Args)
             static if (is(T == InterpolationHeader)) {
                 if (!depth++) {
                     result ~= Span(processedTill, i, true);
                     processedTill = i;
                 }
             } else static if (is(T == InterpolationFooter))
                 if (!--depth) {
                     result ~= Span(processedTill, i + 1);
                     processedTill = i + 1;
                 }
         return result;
     }();

     auto execi(Args...)(Sqlite db, InterpolationHeader header, 
Args args, InterpolationFooter footer) {
         import std.conv: text, to;
         import arsd.sqlite;

         // sqlite lets you do ?1, ?2, etc

         enum string query = () {
             string sql;
             int number;
             static foreach (span; segregatedInterpolations!Args)
                 static if (span.topLevel) {
                     static foreach (T; Args[span.i .. span.j])
                         static if (is(T == 
InterpolatedLiteral!str, string str))
                             sql ~= str;
                         else static if (is(T == 
InterpolatedExpression!code, string code))
                             sql ~= "?" ~ to!string(++number);
                 }
             return sql;
         }();

         auto statement = Statement(db, query);
         int number;
         static foreach (span; segregatedInterpolations!Args)
             static if (span.topLevel) {
                 static foreach (arg; args[span.i .. span.j])
                     static if 
(!isInterpolatedMetadata!(typeof(arg)))
                         statement.bind(++number, arg);
             } else // Convert a nested interpolation to string 
with `.text`.
                 statement.bind(++number, args[span.i .. 
span.j].text);

         return statement.execute();
     }
     ```

     Here, we just invoke `.text` on nested istrings. A more 
advanced implementation would allocate a buffer and reuse it. It 
could even be `@nogc` if it wanted.

3.  DIP1036 appeals more to me because it passes rich, high-level 
information about parts of the string. With DIP1027, on the other 
hand, we have to extract that information ourselves by parsing 
the string character by character. But the compiler already 
tokenized the string; why do we have to do it again? (And no, 
lower level doesn’t imply broader possibilities here.)

     It may have another implication: looping over characters 
might put current CTFE engine in trouble if strings are large. 
Much more iterations need to be executed, and more memory is 
consumed in the process. We certainly need numbers here, but I 
thought it was important to at least bring attention to this 
point.

4.  What I don’t like in both DIPs is a rather arbitrary 
selection of meta characters: `$`, `$$` and `%s`. In regular 
strings, all of them are just normal characters; in istrings, 
they gain special meaning.

     I suppose a cleaner way would be to use `\(...)` syntax (like 
in Swift). So `i"a \(x) b"` interpolates `x` while `"a \(x) b"` 
is an immediate syntax error. First, it helps to catch bugs 
caused by missing `i`. Second, the question, how do we escape 
`$`, gets the most straightforward answer: we don’t.

     A downside is that parentheses will always be required with 
this syntax. But the community preferred them anyway even with 
`$`.