Discussion Thread: DIP 1036--Formatted String Tuple Literals--Community Review Round 1

[Steven Schveighoffer]

On 9/9/20 5:41 PM, Paul Backus wrote:
> On Tuesday, 8 September 2020 at 10:59:31 UTC, Mike Parker wrote:
>> This is the discussion thread for the first round of Community Review 
>> of DIP 1036, "Formatted String Tuple Literals":
> 
> As per the rules, I'm responding here to Steven's post from the Feedback 
> thread.
> 
> Steven Schveighoffer wrote:
>> First, thanks for your opinion. We value all the opinions of people in 
>> the community, especially long-standing members with a lot of 
>> experience with D such as yourself.
>>
>> However, this entire post has no actionable or specific items, and 
>> really should have been added to the discussion thread instead. I'm 
>> responding to it here because in the past I have had cases where I 
>> posted feedback and got no response or changes, and it irked me.
> 
> Thanks for replying. The specific, actionable item in my post is: 
> withdraw the DIP.
> 
> This kind of feedback ("related to the merits of the proposal rather 
> than to the contents of the DIP") is explicitly permitted in Community 
> Review by the Feedback thread rules, which is why I posted it there 
> instead of in the Discussion thread.

I did not realize that was specifically in the rules! Sorry.

> The reason I gave that feedback, instead of more detailed feedback about 
> the DIP's contents, is that I sincerely believe DIP 1036 cannot be 
> salvaged. I have a lot of respect for you and Adam, and would hate to 
> see you waste your time and effort on a proposal doomed to failure. I 
> think you (or anyone else who wishes to take up the string-interpolation 
> torch) would be much better off discarding DIP 1036 and starting a new 
> proposal from scratch.

I disagree completely. This DIP is on the right track. And I'm happy to 
"waste time" if it means it gets a fair shake at inclusion. It costs me 
nothing to have it be judged.

> Regarding the specific points you replied to:
> 
>> The interface is actually simple to use. Without specific concerns, 
>> it's hard to address these comments further.
> 
> If the interface is simple to use, why does the DIP anticipate that new 
> users will have so much trouble using it that they'll need "helpful 
> hints" from the compiler, and possibly even a link to a web page (!), to 
> get it right? I quote:

We do not anticipate much trouble at all for using the feature in the 
DIP. In fact, the opposite is true -- using the dip is really easy if 
you want a string, just append .idup. How is this difficult to 
understand or use?

However, we acknowledge that users who come from a non-D background and 
languages that do not have the vast compile-time introspection 
capabilities that D has (maybe the language they are used to only has 
string interpolation, and not formatted tuples), will try something that 
will not compile. Having the compiler teach them the correct way is not 
a bad thing, nor is it admitting an interface issue. It's an explanation 
of how to recreate the behavior they are used to in D.

It's no different from other suggestions that the compiler does (e.g. 
"writeln not found, perhaps try importing std.stdio").

> 
>> An i"" string is not likely to be compatible with existing D string 
>> functions and variables. The compiler SHOULD provide helpful hints in 
>> error messages to help new users understand how to properly use the 
>> new feature. This message MAY link to a web page to educate the user 
>> on resolving the error.
> 
> To me, this does not pass the laugh test. [1] If you anticipate this 
> being hard enough to get right that new users cannot do it without 
> extensive hand-holding, maybe the problem is not with the users, but 
> with the proposal itself.

We don't. It will be easy to get right. If they get it wrong, they will 
be taught and then get it right. It's not that hard.

> 
>> This measure of "complexity" is arbitrary, and has no bearing on the 
>> validity of the DIP. in fact, many already-accepted DIPS have 
>> relatively similar ratios of description and rationale.
> 
> Again, I quote:
> 
>> ### Justifications
>>
>> The **complexity** of the format spec may seem superfluous, however it 
>> serves four key roles:
> 
> Emphasis added. If you want to nitpick the wording here, take it up with 
> your co-author.

The DIP does not have high complexity. It's straightforward, and 
understandable.

The complexity you quote has nothing to do with the DIP, but is 
referring to how a custom type with more features is more complex than a 
simple string. Again, this is to compare with DIP1027, where a string 
was used as the format specifier. A custom type is obviously more 
complex than a string, and so the DIP explains why we did it.

> 
>> One thing to note is that this DIP has a prior very similar DIP, but 
>> just used a string for the formatting specification. The Justification 
>> section is ENTIRELY devoted to explaining why this DIP does not use 
>> that. In fact, we can remove the entire section, and it doesn't change 
>> anything about the DIPs features or benefits. But without a 
>> distinction between DIP1027 and this DIP, arguably it would be 
>> rejected on principle (why approve something that was just rejected).
> 
> I think members of this community sometimes give the language 
> maintainers (currently Walter and Atila, previously Walter and Andrei) 
> too little credit. They're not perfect, but generally speaking they are 
> reasonable people with good judgment. Have there been any actual 
> examples of a DIP being rejected on principle for being too similar to 
> another previously-rejected DIP?

I might reject a DIP that had a minor change from a previously rejected 
one, without any good explanation as to why. Framing it as us somehow 
thinking poorly of the language maintainers is somewhat insulting. I 
can't imagine anyone putting up a DIP without context of why the DIP is 
designed how it is.

90% of the DIP is actually exactly the same as DIP1027. It's just the 
format spec type which is different. If we do not explain the design 
change, we risk someone filling in the holes with their own idea of why 
we designed it this way (you seem to have no problem incorrectly 
describing our motivation).

> 
>> This assessment is incorrect. There is an implicit conversion of the 
>> format specification to immutable char * in the circumstance that you 
>> have specified all format information. This makes it compatible with 
>> printf.
> 
> "X is true in the circumstance that Y" is not the same thing as "X is 
> true." :)

I admit that in the case where your call to printf would crash your 
program, we disallow compilation.

> 
> Also, the fact that i"${%s}(foo)" and i"$(foo)" are equivalent when you 
> pass them to `writefln` but not when you pass them to `printf` is 
> another thing that doesn't pass the laugh test. You're not making it 
> easy for me to tell my friends about D.

printf is a C function. It does not have introspection capabilities, and 
so your format specifiers have to be exact. This is not hard to explain, 
especially to people with a C background (they have certainly dealt with 
mismatched format specifiers).

How would your friends understand why you can do writefln("%s", 1) and 
not printf("%s", 1)?

> 
>> Implicitly converting the format spec to string would result in 
>> undoubtedly incorrect function calls, and we address the concerns 
>> quite thoroughly. Even if it is a desired thing, it's not something we 
>> should provide. This is not the same as trying and failing. We intend 
>> misuse of string conversion to not compile.
> 
> I understand perfectly. My criticism of DIP 1036 is not that you have 
> failed at what you set out to do, but that you have set out to do the 
> wrong thing to begin with.

How about you tell me what the right thing is?

> 
> By the way, there's another part of that Scott Meyers talk that 
> addresses the problem of accidentally passing arguments to the wrong 
> parameters. The solution he shows there would also work for the example 
> in the DIP, without requiring any of the elaborate machinery the DIP 
> proposes.

I watched and enjoyed the whole talk. The entire time I thought "yes, 
this is why we designed the DIP to be easy to use correctly, and hard to 
use incorrectly". I have no idea how you come away with the opposite 
conclusion.

The format spec could be a simple wrapper for a string. But we then lose 
all the nice compile-time capabilities such as validating formats at 
compile-time, or generating the correct format string at compile time 
without depending on the compiler to know what that means.

> 
>> I would characterize the DIP as not only trying to address these 
>> issues, but succeeding. printf is supported for valid cases. writefln 
>> will be supported as expected. Conversion to string or immutable char 
>> * is possible with a library call which should be provided. There are 
>> no flaws I can see.
> 
> Adding dedicated overloads of writefln, idup, etc. to deal with 
> interpolated strings might sound like a good solution, but it doesn't 
> scale. The *best-case* scenario is that DIP 1036 creates a bunch of 
> meaningless busywork for library maintainers; the more realistic 
> scenario is that most libraries never get updated at all, and users have 
> to either check every individual function they use for 
> interpolated-string compatibility, or defensively spam .idup everywhere 
> (at which point, you might as well just make interpolated strings use 
> the GC to start with).

How many functions do you suppose could use formatted string tuples? In 
Phobos, which I'd consider a large complex library, I bet there are 
somewhere on the order of 5-10. You think this doesn't scale? You think 
an afternoon of updates ONCE ever, is too much work? Those that just 
accept a string do not need adjustment. One just adds .idup with the 
call and it's done.

Your characterization of defensive programming is incorrect. A defensive 
programmer would explicitly do something because they were afraid the 
compiler would COMPILE their code incorrectly. In this case, the 
incorrect code DOESN'T compile, and the compiler tells them how it can 
be updated. No defensiveness or spamming in sight.

One possible improvement is to follow Sebastian Wilzbach's suggestion to 
try .idup if the call doesn't succeed. This might solve a lot of the 
concerns here, but I don't think it's required for the DIP proposal to 
stand. Having to add .idup to the end of an interpolation isn't 
difficult, and most people will learn it quickly.

> DIP 1027 had its issues, but at least it was simple to explain, simple 
> to use, and didn't require the library and runtime to bend over 
> backwards to support it. DIP 1036, by contrast, is hard to explain, 
> fiddly to use, and requires extensive coordination between the compiler, 
> runtime, and library to implement. In short, it's a step in the wrong 
> direction.

This is ironically the exact opposite. It's easy to use DIP1027 
incorrectly -- printf(i"age : $age") or createWindow(i"Pid is $pid"); 
and hard to use it correctly --  mysqlQuery(i"select * from sometable 
where author = ${?}author and date < ${?}publishedBefore and ... ${?}x 
.. ${?}y .. ${?}z ... ${?}why .. ${?}all ... ${?}these .. 
${?}questionmarks");

Our DIP does not have these problems, and is pretty easy to explain.

-Steve