Tuples, CTFE, and Sliding Template Arguments

[Steven Schveighoffer]

On Saturday, 13 January 2024 at 06:46:54 UTC, Walter Bright wrote:
> On 1/12/2024 8:35 PM, Steven Schveighoffer wrote:
>> On Saturday, 13 January 2024 at 02:16:06 UTC, Walter Bright 
>> wrote:
>>> On 1/12/2024 4:15 PM, Steven Schveighoffer wrote:
>>>> I don't view this as simpler than DIP1036e or DIP1027 -- a 
>>>> simple transformation is a simple transformation.
>>>
>>> Adding extra hidden templates isn't that simple. If a user is 
>>> not using a canned version, he'd have to be pretty familiar 
>>> with D to be able to write his own handler.
>> 
>> Yes, that is intentional.
>
> So you agree it is not simpler :-)

No it is just as simple. I agree that the user should have to 
understand the feature before hooking it. I meant it is 
intentional that you can't "accidentally" hook istring calls 
without understanding what you are doing.

And I don't understand this line of argument to begin with. You 
have to be pretty familiar with D to hook anything:

* operator overloads
* foreach
* toString
* put
* DIP1027
* DIP1036e
* this thing you are proposing

And I'm sure there's more.

>
>> You should not be able to call functions with new syntax 
>> because the parameters happen to match. We have a type system 
>> for a reason.
>
> I proposed in the other topic to type the format string as 
> Format (or FormatString), which resolves that issue, as a 
> string is not implicitly convertible to a FormatString.

This doesn't help, as an enum implicitly converts to its base 
type.

>
>
>>> 1027 is simpler in that if the generated tuple is examined, 
>>> it matches just what one would have written using a format. 
>>> Nothing much to learn there.
>> 
>> In other words: "it matches just what one wouldn't have 
>> written, unless one is calling `writef`".
>
> Yes, it is meant for writef, not writeln.

In none of the proposals I have written or supported, has it been 
meant for `writef`. I don't understand the desire to hook 
`writef` and `format`. The feature to make 1036e hook `writeln` 
is just an easy added thing (just add a toString and it works), 
but is not fundamentally necessary. We could just as easily 
change writeln to handle whatever template types we create.

Hooking `writef` involves adding semantic requirements on the 
library author that are specialized for `writef`, for what 
benefit, I can't really say. You can always create a `writef` 
overload that handles these things, but I don't see the point of 
it. String interpolation isn't aimed at formatting, though it can 
be used for it (as demonstrated).

>
>
>>> The other reasons:
>>>
>>> 1. preventing calls to functions passing an ordinary string 
>>> as opposed to an istring tuple
>> 
>> I don't see how this proposal fixes that. I'm assuming a 
>> function like `void foo(string s, int x)` will match 
>> `foo(i"something: $(1)")`
>
> Yes, we've seen that example. It's a bit contrived. I've sent a 
> format string to a function unexpectedly now and then. The 
> result is the format string gets printed. I see it, I fix it.

Me too. But shouldn't we prefer compiler errors? Shouldn't we use 
the type system for what it is intended?

I've literally left bugs like this in code for years without 
noticing until the actual thing (an exception) was printed, and 
then it was hours to figure out what was happening.

> I can't see how it would be some disastrous problem. If it 
> indeed a super problem, `Format` can be made to be a type that 
> is not implicitly convertible to a string, but can have a 
> string extracted from it with CTFE.

This would be a step up, but still doesn't fix the format 
specifier problem.

> What it does fix is your other concern about sending a string 
> to a function (like execi()) that expects a Format as its first 
> argument.

Right, but this doesn't fix the format specifier problem. You 
seem to be solving all the problems but that one.

>>> 2. preventing nested istrings
>> 
>> Why do we want to prevent nested istrings? That's not a goal.
>
> I mentioned in another reply to you a simple solution.

Without a trailer, this isn't solvable technically. But I'm not 
really concerned about nested istrings. I just meant that it 
isn't a requirement to disallow them somehow.

>>> have already been addressed. The compile time thing was the 
>>> only one left.
>> 
>> A compile time format string still needs parsing. Why would we 
>> want to throw away all the work the compiler already did?
>
> For the same reason writefln() exists in std.stdio, and people 
> use it instead of writeln(). Also, the SQL example generates a 
> format string.

The compiler is *required* to parse out the parameters. It has, 
sitting in it's memory, the list of literals. Why would it 
reconstruct a string, with an arbitrarily decided placeholder, 
that you then have to deal with at runtime or CTFE? You are 
adding unnecessary work for the user, for the benefit of hooking 
`writef` -- a function *we control and can change to do whatever 
we want*.

The SQL example *DOES NOT* generate a format string, I've told 
you this multiple times. It generates a string with placeholders. 
There is no formatting. In fact, the C function doesn't even 
accept the parameters, those happen later after you generate the 
prepared statement.

But also, SQL requires you do it this way. And the C libraries 
being used require construction of a string (because that's the 
API C has). An sql library such as mysql-native, which is fully 
written in D, would not require building a string (and I intend 
to do this if string interpolation ever happens).

Things other than SQL *do not require building a string*.

>
>
>> If you want to call `writef`, you can construct a format 
>> string easily at compile time. Or just... call `writef` the 
>> normal way.
>
> ??

Yeah, I've never cared about hooking `writef`, it's fine as-is 
(well, it's fine if that's what you like). The fact that you have 
to put in `%s` everywhere, it's a klunky mechanism for "output 
this thing to a character stream".

Can't tell you how many times I've written a `toString` hook that 
calls `outputRange.formattedWrite("%s", thing);`. That `"%s"` is 
so ugly and useless. But this is all D gives me to use, so I use 
it.

>
>> Compile-time string parsing is way more costly than 
>> compile-time string concatenation.
>
> I suspect you routinely use CTFE for far, far more complex 
> tasks. This is a rounding error.

Wait, so generating an extra template is a bridge too far, but 
parsing a DSL at compile time is a rounding error?

In my testing, CTFE concatenation is twice as fast as parsing, 
and uses 1/3 less memory.

Not to mention that concatenation is easy. I can do it in one 
line (if I don't really care about performance). The same cannot 
be said for parsing.

So I'd say, the user must understand that he's receiving a 
template, but also does not have to learn how to properly parse a 
specialized unrelated DSL. Format strings are weird, confusing, 
klunky, and less efficient.

>
>
>> Ok. This does mean, for *intentional* overloading of a 
>> function to accept a compile-time first parameter, you will 
>> have to rename the function.
>
> You can overload it with existing functions, or give it a new 
> name. Your choice, I don't see problem.

If the template-parameter version is less preferred, it will only 
be used with an explicit template parameter.

It's not a problem, it just is one more quirk that is surprising.

-Steve