D is our last hope

[Adam D Ruppe]

On Wednesday, 20 December 2023 at 13:40:00 UTC, jmh530 wrote:
> I'm confused by what you mean by this. I can't recall seeing a 
> DIP 1036e specification, just an implementation and examples. 
> YADIP had a specification, but was superseded by 1036e.

Walter incorrectly read the YAIDIP specification. There was a 
hope that by implementing it 
<https://github.com/dlang/dmd/pull/15714>, it'd clear up the 
misconceptions.

After implementing the YAIDIP, as written, I decided to go ahead 
and fix some of its shortcomings, and opened that as a separate 
PR <https://github.com/dlang/dmd/pull/15715> which I called 
1036e, since it is a bit closer in design to 1036, but enhanced 
with ideas from YAIDIP.

The implementations of all three (these two and Walter's own 
DIP1027) ever *very* similar. Even to the point of me and Walter 
using - purely coincidentally - almost exactly the same tricks.

Look at how MixinExp is used in both of these diffs, for example:

https://github.com/dlang/dmd/pull/15715/files#diff-a556a8e6917dd4042f541bdb19673f96940149ec3d416b0156af4d0e4cc5e4bd

https://github.com/dlang/dmd/pull/15722/files#diff-a556a8e6917dd4042f541bdb19673f96940149ec3d416b0156af4d0e4cc5e4bd


I marked it as "FIXME" in mine, figuring it was a quick and easy 
hack to make it work, but surely there was a better way. Then I 
looked at Walter's, and he did exactly the same thing!

The biggest difference between the implementations is Walter's 
says things like:

"// Note: doesn't deal with Unicode code units in identifiers" 
and  "// Note: parsing is crude; not handling parens in literals"

Whereas mine handles all these correctly (as you can see in the 
unit tests included in my PR) thanks to doing lexing in the 
lexer, reusing all that existing machinery instead of crudely 
reinventing it: 
<https://github.com/dlang/dmd/pull/15715/files#diff-b893f1fe9a8dab286d255483a3dd8cd982bc321c655ab2be5f8d14402aeea5be>

But they're still built on the same concepts, and in many places, 
almost the same code! The core of the implementation is quite 
simple: lex the string, using sub-lexers for the interpolated 
elements, then lower to a tuple of the elements. There's even 
comments in the code showing the idea.

A big reason why my diff is bigger is that my PR includes 
user-facing documentation comments: 
<https://github.com/dlang/dmd/pull/15715/files#diff-22dd3c85816b0aa6351d1bc21e72ed410b509f8f7c768db3dd974967751b37e1> which further, along with the half dozen examples of use cases I wrote up (separately, but also including some design rationale <https://github.com/adamdruppe/interpolation-examples/#feature-design-faq>), help to explain what is going on beyond sharing the same conceptual foundation as the other DIPs.

If you want to know how this works, you can:

1) Read the ddoc included in the PR. This is written for library 
authors, not compiler implementers, so hopefully it is accessible.

2) Read the examples repo. This is written for end users, to show 
them of the possibilities.

3) Read the comments in the implementation, which explain what 
the code does in simpler terms than dmd's AST, and the included 
unit tests which show what is and is not expected to work.

4) Compile it and try it for yourself!