Meaningful identifiers and other multi-token keywords

[Quirin Schroll]

On Wednesday, 25 September 2024 at 15:50:20 UTC, Tim wrote:
> On Tuesday, 24 September 2024 at 20:37:36 UTC, Quirin Schroll 
> wrote:
>> I want to suggest moving the parsing of scope guards and 
>> linkages to the lexer, i.e., if the lexer sees `scope`, `(`, 
>> any one of the identifiers `exit`, `success`, or `failure`, 
>> and `)`, that is a scope guard and is treated as a single 
>> token.
>>
>> The same with `extern(C)` – it will never be seen as anything 
>> but a linkage. It’s a multi-token keyword.
>
> I don't think, the lexer would be the right place, because the 
> constructs are still multiple tokens. For example whitespace 
> and comments are allowed in `extern  (  C  ++ /*comment*/ )`.

The whitespace is not an issue. The comments maybe are. But even 
if they were, one option would be to just ban comments in linkage 
attributes and scope guards and not deal with the problem. I 
mean, who would do that, except for a QA tester?

> Unknown languages in `extern(...)` attributes should also 
> produce errors, so future compilers can add them without 
> breaking code.

Officially, it’s implementation defined what’s supported beyond 
`D` and `C`, see 
[here](https://dlang.org/spec/attribute.html#LinkageAttribute). 
The fact that DMD supports `C++`, `Objective-C`, `System`, and 
`Windows` is already an extension.

Considering C++ namespaces, the syntax is quite flexible. 
Essentially, any token soup with balanced parentheses is allowed. 
Maybe C++ was right, there it’s `extern "C"`.

> Consider this example:
> ```
> extern(X) x = 0;
> ````
> Currently `X` is a normal identifier, but in the future it be 
> could another language supported by the compiler. If `(X)` is 
> interpreted as a type, then adding `extern(X)` to the compiler 
> would be a breaking change. For forward compatibility it would 
> be best if `extern(...)` and `scope(...)` are always parsed as 
> whole attributes and not attributes with types in parens. 
> Unknown languages or scope guard identifiers would then produce 
> errors, so future compilers could add them without breaking 
> code.

With Primary Type Syntax, `extern (Type)` can happen by accident, 
yes.

Then, `Type` could happen to be a valid linkage, but even in that 
case, there’s a high likelihood that there’s a parse error down 
the line. (I fact, it might be guaranteed, I couldn’t find a way 
how it’s not.) That is because linkage attributes are not storage 
classes. Unlike `static`, `ref`, etc., `extern(C)` cannot be used 
instead of `auto`.

```d
// Current behavior:
alias C = int;

extern (C) x = 0; // Error: basic type expected
extern (C) auto x = 0; // Good, and `C` can’t be the type of `x`

static extern (C) x = 0; // Error: basic type expected
extern (C) static x = 0; // Good, and `C` can’t be the type of 
`x`, even if it denotes a type

alias Type = int;

extern (Type) x = 0; // Error: Type is not a linkage
extern (Type) auto x = 0; // Error: Type is not a linkage

static extern (Type) x = 0; // Error: Type is not a linkage
extern (Type) static x = 0; // Error: Type is not a linkage

// My implementation:
alias C = int;

extern (C) x = 0; // Error: basic type expected
extern (C) auto x = 0; // Good, and `C` can’t be the type of `x`

static extern (C) x = 0; // Error: basic type expected
extern (C) static x = 0; // Good, but `C` can’t be the type of 
`x`, even if it denotes a type

alias Type = int;

extern (Type) x = 0; // Error: `Type` is not a linkage
extern (Type) auto x = 0; // Error: `Type` is not a linkage

static extern (Type) x = 0; // Error: `Type` is not a linkage
extern (Type) static x = 0; // Error: `Type` is not a linkage
```

 From what I’ve understood in [the attribute 
spec](https://dlang.org/spec/attribute.html), `extern` marks a 
symbol a declaration whereas without, it would be a definition. 
It comes with or implies `export`, or at least implies `static`. 
So, if needed, one can just put that (or any nothingburger) 
between `extern` and `(Type)` and be good.

```d
// My implementation
extern export (Type) x; // ok
extern static (Type) y; // ok
extern @0 (Type) z; // ok
```