D Language Foundation March 2023 Monthly Meeting Summary

[Mike Parker]

The monthly meeting for March took place on March 3rd, 2023, at 
14:30 UTC, and lasted about an hour and fifteen minutes. The 
following people attended:

* Walter Bright
* Iain Buclaw
* Ali Çehreli
* Martin Kinkelin
* Dennis Korpel
* Mathias Lang
* Átila Neves
* Razvan Nitu
* Mike Parker
* Robert Schadek

## The summary

### Dennis
Dennis had nothing for us this time.

### Robert
Robert had nothing other than to say that the compiler is still 
too slow and he still likes D.

### Razvan
__Preview Switches__

Razvan had been looking at the reported issues for 
`-preview=nosharedaccess` and had found only a handful from 
Átila. This suggests that either we have a good implementation 
without many bugs, or it isn't being used very much. Átila 
suggested no one is using it at all. Razvan noted that [one of 
the bugs Átila had 
reported](https://issues.dlang.org/show_bug.cgi?id=23709) was 
that trying to instantiate a shared class resulted in an error. 
If that doesn't work, there may be other issues. He wondered what 
path we could take to get to where the preview becomes the 
default. Other than waiting for people to use it and report bugs 
on it, what can we do?

Átila said he had been trying to get Phobos and DRuntime to 
compile with it, and then once they do, try it on dub packages. 
He said he'd encountered so much noise, he decided to wait until 
the bugs he'd reported are fixed before reporting more.

Iain suggested we should have a pipeline that builds with all 
preview switches turned on by default. Dennis noted that would be 
a problem with preview switches that are broken.

Mathias asked what a good litmus test for previews would look 
like. He'd had problems with `nosharedaccess` and 
`rvaluerefparam` not working when he tried them, then he'd given 
up on previews. Átila didn't know a good litmus test but was 
working on going through the previews and deciding on either 
making it the default or scrapping it, especially the ones that 
had been there for years, so that they aren't left in limbo. I 
suggested that at some point on the other side of our 
organizational build-up, we establish a policy for periodic 
reviews of previews.

Razvan said his main question is whether preview switches are 
serving their intended purpose. They were started so that people 
could test new features before they are officially part of the 
language, but in practice, it seems they are rarely used. He 
thinks we should have a different mechanism to ensure new 
features are tested before they are merged into a compiler 
release, then once they are merged they're ready for use. The 
preview switch seems to him like a graveyard for features.

I said that seems like a communication issue. Right now, we just 
throw a feature behind a preview switch and do nothing beyond 
that. I suggested we come up with a staged process that motivates 
people to use the previews. Robert would go a step further. He 
thinks there should be a burden on anyone wanting to merge a new 
feature to test the feature with the list of dub packages that 
get tested with new compiler releases, then investigate any 
errors. The top 30 or whatever packages should compile both with 
and without the preview before it's allowed in. Átila agreed.

Mathias said the idea of the preview is fine and it can work. We 
just need to take it seriously. He said he agrees with Robert. 
When he implemented `-preview=in`, he did exactly what Robert 
suggested. [He enabled it by 
default](https://github.com/dlang/dmd/pull/11632) and then 
examined what it broke. Generally, previews are implemented with 
half-baked code and we don't test if they work. We need to ensure 
that people implementing previews do the work correctly.

Walter said we could submit a pull request for each preview 
feature to turn it on by default. We don't have to merge them. 
That would ensure that it's run through BuildKite (which tests 
several dub packages), and we'll have links to everything that 
breaks. Then we reach out to the maintainers of broken packages 
to help upgrade their code.

In the end, we decided that Razvan and Átila will gather up 
issues with all of the preview switches and we can decide their 
fates at future meetings.

__ at system and pure__

Razvan said that once in a while there are issues with `@system` 
and code and `pure`. He says that trying to do system-level 
things, like allocating memory without initializing it, can clash 
with purity, whereas in `@safe` code purity will usually be 
compatible. But then you can do something like void initialize a 
stack variable in `@safe` code and return it, and now you have 
this clash between two concepts. Looking forward to the day when 
safety and DIP1000 become the default, he was wondering if 
there's a way we can enforce that safety and purity don't go 
along together.

This led to a side discussion about why people would do things 
like allocating memory without initializing it. Robert said that 
whether they should or shouldn't be doing it, the language allows 
it. Either the language should prevent it, or it should work, or 
give useful error messages. Just telling people not to do it 
isn't practical.

Razvan brought up `std.experimental.allocator` as an example. 
Something like `Mallocator` can't be pure, but allocators like 
that still need to be used in pure code. Walter said that 
fundamentally a storage allocator cannot be pure, and the 
language is correct in its behavior. If you want to call an 
allocator from pure code, then you're going to have to do 
something dirty in your code. And that's okay. The language 
shouldn't support by default making a storage allocator pure. 
It's doing its job, saying you can't do this because you're 
changing state. So you have to fake it by doing a dirty cast in 
there somewhere. The reason why D supports system code is so you 
can do dirty things when you have to. It's the same with `const`. 
D does not support logical const, because logical const is not 
const. It's not enforceable. That's a feature. If you want 
logical const, you have to do something dirty to get it, and 
that's appropriate.

Walter also said that whether void initialization should be 
allowed in pure code is a different question. Void initializing 
an int and returning it isn't pure, is it? So maybe we should 
disallow void initializations in pure code. He thinks that's a 
reasonable proposition. Átila agreed and added that system calls 
are by their very nature not pure. It's okay that they're not 
`pure` because it would be lying to say otherwise. Walter agreed. 
System calls usually change the system. There's no way the 
compiler can tell. Maybe some really are pure, and we could 
individually mark them as such. But in general, you just have to 
cheat on it. He likes that D's notions of const and pure are 
stringent.

Dennis had what he called a "wild proposition" that he wasn't 
sure he agreed with himself: what if we get rid of `pure`? People 
seem to dislike attributes, and he hasn't met anyone saying "Wow, 
D has purity and it saves all my code". What does it buy us? 
Átila said that he says that, so now Dennis has met someone who 
does. Dennis said he uses `pure` as well, but he never feels like 
he really needs it.

Átila said that at Symmetry, because people didn't use `pure` 
when they should, there's missing functionality that wouldn't be 
missing had they used it. He cited an example from the SIL 
project (Symmetry's internal DSL). The TL;DR: a feature to look 
up a function name that involves calling functions to get their 
outputs---you don't want to call a function that's going to 
format your hard drive. If a D function is marked `pure`, then 
it's also `pure` in SIL and therefore callable in this case.

Walter noted that pure functions also can be made multi-threaded 
without risk. They live on their own island that doesn't interact 
with anything else. Átila said you can also initialize 
`immutable` things from pure functions, which is also very handy. 
Robert said that in his D code at Symmetry, there are a few 
functions that are pure because it was simple to mark them. 
Nothing else is pure or immutable. For every example, he bets we 
can find a counterexample.

There was more discussion in this vein, then Walter said that 
`pure` can be difficult to use but should be a goal. A good 
principle in programming is that everything a function does 
should come through the front door. If you can make a function 
pure, then you know that's the case. You know certain things 
about it: it has no side effects, it's thread-safe, etc.

Dennis's motivation for bringing this up was complaints about 
attribute soup. This led to a short discussion that meandered 
through tangential topics and isn't worth summarizing here.

### Iain
Iain reported that [he'd released 2.102.2 on March 
1st](https://forum.dlang.org/thread/mfztadxwqhcoabwuqfuo@forum.dlang.org) and [followed that with the 2.103.3 beta](https://forum.dlang.org/thread/enokkwxrjoerezuajzxd@forum.dlang.org) which had been cut and merged into the stable branch. So for the next two weeks, upstream dmd development should be focused on stabilizing the stable branch before the release candidate, so preferably all bug fixes should target stable and any open bug fixes should be retargeted to stable. Once the RC was released ([as it was on March 16](https://forum.dlang.org/post/fmzqqpzrzhoheyluelzv@forum.dlang.org)), then things could go back to normal where only regression fixes target stable.

On the GDC front, a major new GCC version is released every year, 
typically in May or the end of April. So the speed of GCC 13 
development was winding down. Iain was following the dmd stable 
branch, so the GDC 13 release should be around D 1.103.1. At the 
moment, he had a couple of issues to resolve. He'd had a few 
missing template symbols when running the test suite, probably 
due to a change in the template emission strategy. The reason it 
affects him is that he builds DRuntime and Phobos using separate 
compilation. He hadn't yet found the time to resolve that.

Martin asked which front-end version the template emission bugs 
showed up with and whether it was DRuntime or Phobos showing 
them. Iain said it was in mainline Phobos after January. They 
come back as unresolved because of link time failures. He's aware 
he's probably the only person who builds DRuntime and Phobos 
using separate compilation. Martin said he only uses separate 
compilation for unit tests now, otherwise, memory usage explodes. 
Iain said he compiles unit tests separately as well. It's very 
helpful to compile each unit test in isolation, as you can amp up 
the parallelism on the make side, and one failed unit test module 
doesn't prevent the testing of other modules.

Iain said there had been a few codegen glue changes as well, 
which had been introducing regressions. Because a change like 
this is in the backend of dmd, he can work out how it's 
implemented and figure out what he needs to be doing to implement 
it for GDC. Then once he does, he spends 15 minutes figuring out 
how to break it. Then the first test he writes shows that there's 
run-time corruption.

He'd been constrained on time for the past few months, but he did 
hope to be on 2.103 by the end of March, regardless.

GDC documentation [was now online at 
gnu.org](https://gcc.gnu.org/onlinedocs/gdc/). He said there were 
probably still a few gaps left to fill.

He had been supporting C++ developers in GCC porting D to their 
platform. They'd raised some points, some of which can be easily 
explained, e.g., the distinction between struct and class, but 
others are more interesting. One example that turns them off is 
that we still generate virtual function tables for `extern(C++)` 
classes that only contain fields and no functions. Another is 
that D doesn't allow dashes for identifier symbols, so that e.g., 
`foo-bar.d` doesn't work as a module name, whereas in C++ you can 
`#include foo-bar.h`. They find it very surprising to have file 
name constraints like that. He's not sure there's anything we 
need to do about it, but it's sometimes interesting to get 
opinions from someone who thinks differently when trying to use 
our language.

Dennis noted that the dashes in file names came up in ImportC, 
specifically when trying to import the Wayland headers. Walter 
said you just write a wrapper for it. You make another header 
file that just `#include`s the problematic one, or put the 
problematic file on the command line and use `-mv` to rename it. 
He doesn't see it as a significant problem. Iain says that you're 
then just working around a compiler that's working against you. 
Walter said neither of the two ways to deal with it is onerous, 
so he doesn't see that we need a language change for that.

I noted that the whole point behind ImportC is that it "just 
works". That's the goal. Walter agreed. I said that the people 
who *need* that "just works" aspect aren't going to be aware of 
the two workarounds. Walter said it should be documented. Átila 
said he didn't even know `-mv` existed. Martin added that if 
you're dealing with a small set of files, the wrapper workaround 
is fine, but if it's a large code base with numerous problematic 
header file names, it doesn't work.

Walter said he has so many issues with ImportC that he considers 
this one low priority and not a blocker. ImportC wasn't compiling 
the standard header files. That one was a blocker because there 
was no workaround, and those should have priority. The file names 
problem has a workaround, so isn't a blocker. In any case, 
ImportC is going to need a valid identifier to use as a module 
name. Maybe something like `pragma(mangle)` could help.

Iain had a sudden thought: do we have a document on how to write 
D code coming from a C++ mindset? Walter said things like this 
should go into [the page on interfacing with 
C++](https://dlang.org/spec/cpp_interface.html). It's an old 
document that's probably in desperate need of modernizing. Dennis 
and Mathias both linked [the C++ to D 
page](https://dlang.org/articles/cpptod.html) in the chat, and 
Mathias suggested it's a better place for the C++ mindset stuff 
as the interface page is more about the language features.

I mentioned [a book called 'C++ for Java 
Programmers'](https://www.amazon.com/C-Java-Programmers-Timothy-Budd/dp/0201612461/). A 'D for C++ Programmers' book might be nice if someone with intimate knowledge of both had time to write it. Walter suggested that D might have the same issues Java has when interfacing with C++, so that book might serve as a guide for what the D version would need to cover.

### Ali
Ali didn't have anything to report and just wanted to say hello 
to everyone.

### Átila
Átila reported he'd had some internal discussions at Symmetry 
about DIP1008 because of some problems they'd encountered 
interacting with C#. Because D and C# each have a GC, they don't 
want to just hand over memory allocated from one GC to the other 
side, so they'd used `@nogc` everywhere. They ran into problems 
with exceptions. DIP1008 didn't work for them, and they ended up 
having to manually deal with the reference counts. The root of 
their problem is that they're interacting with C# and the D 
library they're using is quirky, so he's not sure if there's 
anything actionable for us.

Martin added that they'd be quite happy if `@nogc` checks were 
skipped for throw expressions. Especially with the recent change 
to not use the GC in stack trace generation. He wanted to 
experiment with this to see if it helped with their use case at 
Symmetry.

Átila had recently been talking to Iain about ImportC and needed 
to talk to Walter about it later. He's also been talking with 
Mathias about dub. Átila thinks it makes sense to take the stuff 
he'd done in reggae and put it in dub. Most people are going to 
use dub, and that's what we point them to, so it makes sense to 
put reggae's functionality there.

He repeated that he'd been going through the preview switches to 
decide their fate. If they're new, that's fine, but things like 
`rvaluerefparam` and `nosharedaccess` that had been there for a 
long time need to either be made the default or dropped. He'd 
started by trying to compile things with `nosharedaccess` and 
immediately encountered a bunch of bugs.

Finally, he asked what everyone thought about including Phobos in 
the dmd/druntime mono repository. He said he'd seen no good 
reason not to [in the forum thread he'd 
started](https://forum.dlang.org/thread/kdgcznersjetunkvorzh@forum.dlang.org). Iain said that from his perspective, DRuntime and Phobos are the same thing anyway. Mathias voiced support for it. Robert said that if it happened, the first thing he'd do is replace some instances of `filter` with the Phobos version, as he thinks we should be using Phobos in DMD anyway. (Note that no final decision was made here.)

### Walter
__Standalone lexer__

Over a series of pull requests, Walter had gotten closer to 
making the lexer a standalone module, meaning it should be easy 
for someone who wants to build something like a syntax 
highlighter in their editor---they'll be able to just pull in the 
lexer and nothing else. Part of that included having the error 
handler come in through the front door. He knows that caused 
unhappiness with some people. He was sorry about that, but the 
result was worthwhile. It works nicely, and he'd already done 
some pull requests to use error sinks in other parts of the code. 
So far it's working well. It makes the lexer's unit tests 
simpler, as it's much easier to mock the lexer when everything 
comes through the front door instead of global variables on the 
side. It wasn't there yet, but getting it there was on his list.

__ImportC__

ImportC was slowly grinding along. His priority was getting all 
the standard header files to compile on all the platforms. A lot 
of the system `.h` files on the various platforms have made a 
feeble attempt to be usable by different compilers but fail 
miserably, and each one fails in its unique fashion. They'd be 
easy to fix if he could edit those files, but of course, he 
can't. So he's left with trying to come up with creative 
workarounds. That's high on his list. Anyone trying to use 
ImportC by importing system header files is just going to quit 
and write off ImportC as unusable when it fails.

__Template lowerings in CTFE__

One problem he'd discovered was that the semantic routines take 
several constructs and lower them to templates. They do that even 
when running in CTFE. Those templates can generate a lot of 
bloat. In CTFE, it should not do the lowering. CTFE finds it much 
easier to interpret the higher-level code. Upon investigation, he 
found that CTFE reverses some of the lowerings so that it can 
interpret them. If you're doing certain things in CTFE like array 
concatenation, you're suffering from a lot of invisible template 
bloat. That needs to be fixed. He was about halfway through it. 
He was looking at all the lowerings and rewriting them so that 
the lowering isn't done in CTFE. He hopes that this is another 
step in reducing the problems with memory consumption and slow 
CTFE speeds. He also hoped to be able to remove the "lowering 
unwinding" that goes on in the interpreter.

Iain noted that in [the PR rewriting the  `_d_newclass` runtime 
hook to a 
template](https://forum.dlang.org/post/fmzqqpzrzhoheyluelzv@forum.dlang.org), he'd noticed the same thing Walter had just described. He got the author to change things such that rather than doing this lowering->unlowering->relowering thing, the rewrite is put into a field called `lowering`, leaving the original node intact. When you go through CTFE, you just work on that node specifically. Then once you're in codegen, you hit that node and go into the `lowering` field. Walter said that wasn't done with the other rewrites, but Teodor Dutu and Razvan plan to go back and change them (here's [an example of work done toward that end](https://github.com/dlang/dmd/pull/14985)). Walter said this will also be a big win for allowing more expansive use of CTFE in BetterC.

__Resolving disagreements__

One question that had come up recently was how to resolve 
disagreements between contributors. For example, what should a 
reviewer do if Walter wants to merge a PR over objections from 
Iain or Martin? Walter declared that Iain had the last word on 
anything involving GDC. He's in charge of it, he knows what he's 
doing, and Walter has full confidence in him. The same goes for 
Martin---he has the last word on how things happen in LDC. As far 
as D goes, when there's a dispute and there's no meeting of the 
minds, Walter has to make a decision. That's his responsibility. 
In that case, if he asks a team member to do something on the D 
side that goes against a position taken by Iain or Martin, then 
Iain and Martin need to direct their ire at Walter and not the 
team member. He's responsible for taking the heat for his 
decisions. No team member should be held accountable for that. 
(To be clear, neither Iain nor Martin has directed any ire at any 
team member. This came up in the context of reviewing a PR about 
which Walter and Iain disagreed, and the reviewer was concerned 
about upsetting one or the other.)

__Focusing on bugs__

As decided [in the January 
meeting](https://forum.dlang.org/thread/sqhogpnlobltogngobpd@forum.dlang.org), Walter had continued and was continuing to focus on bugs rather than new features. Sum types had come up in a wishlist someone had sent to me. Walter said that he would like to have sum types. It's a modern feature people expect to have and we ought to do them. He posted [a draft proposal back in November](https://forum.dlang.org/post/tm48m8$u4g$1@digitalmars.com), but that's on the back burner for now. We've got enough on our plate.


### Me
__Random stuff__

I reported that Robert had come into the Jitsi instance 20 
minutes early, and he and I had discussed the state of his work 
toward the Bugzilla to GitHub transition. He'd told me the script 
was just about ready and he could send it to me very soon. Given 
that we were both planning vacations over the coming weeks, we 
decided to hold off until the middle of April.

I asked everyone to consider emailing me some of their issues so 
I could incorporate them into the gripes and wishes dataset I was 
collecting. We discuss several different issues in these 
meetings, but it would be nice for them to send me their pet 
peeves so they don't get lost.

I informed everyone that I had recently received an email from 
someone at a company using D in multiple production projects: a 
360-degree photo booth, karaoke on a Ferris wheel, and a 2mx2m 
interactive wall. All of the projects use a telegram bot written 
in D that reports logs and diagnostics. The karaoke project uses 
ImportC.

__Gripes and wishes__

I gave an update on the Gripes and Wishes campaign. I'd received 
some positive remarks about the decision to focus on stability 
and robustness after I noted it in [the summary of the January 
meeting](https://forum.dlang.org/post/sqhogpnlobltogngobpd@forum.dlang.org) (see the section titled 'John and Amaury'). Most people sent me two or three items, some actionable, some more general. I noted that the biggest exception was Timon, who sent me had sent me a long list of specific items (the longest by far!). I intended to split everything up into two reports: one for the actionable things and one for the general.

The biggest surprise for me here was that I had expected a 
handful of commonly hated or desired actionable items to float to 
the top, but that hadn't been the case as far as I could see so 
far. A few items may have been repeated once or twice, but almost 
everything was unique. Nothing was overwhelmingly common.

__Quarterly and monthly meetings__

Our next meeting, scheduled for April 7, was going to be a 
quarterly. I wanted to start bringing more people into the 
quarterlies, as shops were using D in production who weren't 
represented. As such, I didn't think it was feasible for us to 
continue to divide the quarterlies into two parts (we had always 
let the industry reps go first, and then they could stick around 
as we moved into our regular monthly meeting business after). The 
size of the quarterlies had already expanded to a degree that 
they could go quite long, and more people would just make it 
longer. I proposed that we split the quarterlies into two 
separate meetings: the quarterly on the first Friday of the month 
for the industry reps, and then a regular monthly meeting on the 
following Friday. Everyone agreed.

__UPDATE__: I had hoped to get the gripes and wishes mostly 
organized during downtime on my trip to the States at the end of 
March, expecting that I would find myself with plenty of it. It 
didn't turn out that way. I'm going to get it completed as soon 
as I wrap up the April meeting summaries.


### Martin
Martin said he'd been busy with LDC as he had to catch up with 
the latest DMD releases. He thinks that's always good as he has 
CI test coverage for platforms that aren't tested by DMD, so as 
always he'd found some minor regressions. Those were fixed in DMD 
2.102.2.

He'd found some regressions related to the lowering of runtime 
hooks to templates. For example, the new lowering for array 
concatenation meant that CTFE string concatenation didn't work 
for targets like WASM because the new lowering required some 
standard C bindings. Luckily, it was simple to fix. In general, 
these lowerings need to be carefully designed in this regard.

Átila wondered if we could add LDC to the CI somehow so that we 
catch stuff like this beforehand. Martin said there was no way at 
all.

He's quite happy about the new 'lowering' field that Iain 
mentioned. They'd been spinning around the idea that there could 
be a fourth semantic phase where the more high-level AST is 
lowered to the actual codegen AST, and this seems like a step 
toward that. Or something similar at least.

 From the perspective of the glue layer for LDC, one of the bigger 
changes in the 2.102 release was the change regarding vector 
comparisons. They now yield a vector mask based on the front-end 
AST result. He was surprised as the change didn't make it to the 
changelog. This is a pain with floating-point vectors. The 
bitwise operators aren't available for floating-point vectors, so 
he had to do a lot of reinterpret casts to treat them as integer 
vectors for the bitwise ops. In that regard, GDC and LDC treat 
casts of vectors (e.g., casting a `float4` vector to `int4`) as 
reinterpret casts, but with DMD that's a conversion. That's an 
important difference. Another thing is that identity comparisons 
aren't available in DMD or GDC at the moment because they're 
prevented in the front end, but LDC does support those and lowers 
them to yield a scalar boolean.

### Mathias
Mathias wanted to get to a point with dub that overcomes the bug 
where it brings in all potential dependencies. When a dependency 
is part of a subconfiguration, but a different subconfiguration 
is enabled, that dependency shouldn't be pulled in at all. That's 
probably the most reported dub issue. As an example, we can't 
have test-only dependencies at the moment, as that will 
contaminate everything else. He'd been working with that as well 
as some speed improvements prompted by a recent discussion with 
Átila.

The other thing is that he wanted to get `-preview=in` enabled by 
default. He was fairly happy with the way it works. He'd been 
using it for years in production and it was working exactly as he 
expected. A few minor things had been missing to make the 
transition easier. One was the inference of storage class on 
function literals, a bug he'd fixed a few days before the 
meeting. He was still trying to figure out the best deprecation 
path for a few things, but getting closer to getting to the state 
where we can enable the feature.

He said he wants to see colors in his stack traces, so he had 
been working on the demangler so that we'll be able to colorize D 
symbols straight from the demangler. If it doesn't work, then it 
doesn't work, but it will make the demangler code better for sure.

## The next meeting
The next meeting was a quarterly and took place on April 7th at 
14:00 UTC. This was the first case following our decision to 
split out a monthly from each quarterly. April's monthly meeting 
is scheduled for April 14th at 14:00 UTC.