D Language Foundation July 2024 Quarterly Meeting Summary

[Mike Parker]

The D Language Foundation's quarterly meeting for July 2024 took 
place on Friday the 5th at 15:00 UTC. It lasted for about an hour.

Our quarterly meetings are where representatives from businesses 
big and small can bring us their most pressing D issues, status 
reports on their use of D, and so on.

## The Attendees

The following people attended the meeting:

* Mathis Beer (Funkwerk)
* Walter Bright (DLF)
* Johan Engelen (Weka)
* Luís Ferreira (Weka)
* Martin Kinkelin (DLF/Symmetry)
* Dennis Korpel (DLF/SARC)
* Mario Kröplin (Funkwerk)
* Mathias Lang (DLF/Symmetry)
* Mike Parker (DLF)
* Carsten Rasmussen (Decard)
* Robert Schadek (DLF/Symmetry)
* Bastiaan Veelo (SARC)

## The Summary

### Carsten

Carsten said Decard had been experimenting a lot with 
WebAssembly. They'd stubbed out DRuntime until it worked. They 
had some of their library functions running in a browser and had 
tried different virtual machines. They wanted to do more work on 
that. They had a lot of issues with mobile because different 
chips worked in various ways. They wanted to get their entire SDK 
working on WASM, but it would probably take a year to complete 
because there was a lot of stuff moving around in WASM.

Robert asked if they'd gotten the GC working. Carsten said they 
hadn't focused on that. They were just calling functions. They 
didn't know how to write a GC, so they were hoping to use the 
WASM GC eventually.

He said one thing that had been annoying was that the WASM 
interface was basically POSIX, but the compiler didn't treat it 
as such for versioning. He'd had to implement a lot of versioned 
dummy code to get it to work. He would like to see the compiler 
recognize WASM as POSIX.

Robert said he'd seen that the standardizing body had accepted a 
proposal for GC. Carsten said they'd also accepted 
multithreading. He thought WASM was moving into virtual machines 
and such, so it would eventually be a VM for both in and out of 
the browser. Robert said it was a really nice interface.

### Bastiaan

Bastiaan said he was still working on SARC's D translation. Now 
and then he encountered a problem that was difficult to solve or 
difficult to understand. That was his life, but there was nothing 
major to bring up here.

### Mathis Beer

Mathis said that for the first time ever Funkwerk was using a 
patched version of the compiler because the rebindable range 
`destroy` bug was so horrible. He said that wasn't a new thing, 
though, because it was mostly his fault. He thought the `destroy` 
function should just be deprecated. It just did so much harm, but 
he didn't feel it was debate-worthy.

Dennis asked if he meant the general core `destroy` function. 
Mathis clarified that he meant the `Object` version. He said it 
had this issue where when it was dealing with a value type it 
would do something to the value, but when dealing with a class it 
actually destroyed the class value, not the reference but the 
actual object itself, in a way that was very hard to notice. 
Sometime later you'd notice that your vtable pointer had become 
zero, so you had to know what that meant and find out where it 
happened.

He thought it was almost never the function you wanted to call in 
the generic case. He couldn't think of a single case for it. He 
found it bewildering.

Walter thought the purpose of `destroy` was so you didn't have to 
wait for the GC to collect an object. You could just say you were 
done with it. Mathis said no, it didn't even deallocate.

Walter asked why it mattered if it set the vtable to zero if you 
weren't going to use it anymore. Mathis said the problem was that 
he was using it in a generic context and wanted to call the 
semantically appropriate destruction operation for a value. For a 
struct, that meant calling the destructor because the value's 
lifetime was ending. But when the lifetime of a class reference 
ended, nothing should happen because there might be other 
references to that class instance. But what actually happened was 
that the class instance was destroyed no matter how many 
references remained.

Walter said that was the point of `destroy`. Mathis said it 
didn't do that for any other type. If he had a struct lying 
around somewhere and passed a copy to `destroy`, the original 
version wouldn't be affected. Dennis said that was true, but if 
you had a pointer to that struct, then the pointer would be 
dangling. Mathis said he didn't actually know what `destroy` did 
with pointers. He thought it did nothing.

Walter said if you were to destroy something, then there should 
be no more references to it. It was up to you as the programmer 
to ensure there were no more references to it. Mathis said in 
this case he was using it to compensate for lifetime management 
of [a data object in 
rebindable](https://code.dlang.org/packages/rebindable) where he 
had to take charge of the lifetime. He thought he'd call 
`destroy` to end its lifetime.

Walter said, no, you called `destroy` when you as the programmer 
know that was the end of the targeted value, which was dependent 
on the specific type. So if you were using it in a generic 
function, if it was a struct, then it was going to be a copied 
value so it made sense to say you wanted to end the lifetime of 
that value. But if it was a class, the copy constructor was 
literally just making a copy of the reference. If you called 
`destroy` on that, you were saying that there were no more 
references to it.

Mathis said that was his point. The behavior was different 
semantically. He wasn't calling it on a class specifically. He 
was calling it on a `T`, an arbitrary template parameter. Walter 
said that by doing that he was saying there were no more 
references to that `T` instance. But here in the meeting he was 
saying he had more references and they were going to go bad, but 
that was not the point of `destroy`. The point was that the 
programmer was saying there were no more references.

Mathis said the behavior made sense for classes and made sense 
for structs, but it didn't make sense for both classes and 
structs. It was different. Walter said that was the way class 
objects worked. Calling `destroy` was unsafe.

Mathis said that basically, it didn't make sense to be 
overloaded. There should be a `destroy` and a `destroyClass`. 
Walter again repeated that this was the point of the function: 
calling it meant you were indicating there were no more 
references.

Martin said the problem with classes was that you could never be 
sure that the reference you were destroying was actually the 
final reference. He suggested we should have a separate finalizer 
for classes. It would mean breaking changes, but we could add a 
finalizer for class objects and then in an edition, we could 
change the semantics of `destroy`.

Walter thought it was a good idea, but said the trouble with 
having a separate `finalize` and `destroy` was that people would 
mix them up, wouldn't understand the difference, would use the 
wrong one, and then be unhappy. It was inevitable.

He said one thing Mathis could do was to wrap the call to 
`destroy` with "is this a class". Mathis said he'd been doing 
that. He couldn't ever remember calling `destroy` without 
wrapping it like that. Walter said okay, it was working the way 
it was designed to work, and he didn't think we could change that 
behavior.

Luís thought one of the problems here was special cases for 
classes. Not only `destroy`, but a lot of stuff was special-cased 
for classes because they were just pointers to hidden objects 
plus a vtable. Another thing was that classes implied the 
presence of the GC, but you could also allocate them without the 
GC. The existence of the special cases was why Weka didn't use 
classes. Most of the cases could be solved with structs.

Walter said Luís was right that classes and structs were 
different from each other. Algorithms that ignored those 
differences were going to lead to trouble. Luís understood that 
but felt the design could have been more similar to structs 
without the differences that always need special cases in generic 
programming.

Walter said that in C++ you had to special-case for references. 
When you had value types and reference types, they were going to 
be different. He didn't think there was any magic way to fix 
that. Luís said at this point he didn't think so either.

Mathis agreed. He said he really liked the class design in D. The 
standard range algorithms worked fine with classes as well as 
with structs when written in a straightforward way. The only 
function he had a problem with was destroy. He felt that if he 
had a generic function that required him to use a `static if` to 
call a function, then in fact it was two functions that happened 
to be named the same.

Walter said that was a good point, but if you were using value 
types, you should be relying on RAII to do the destruction, not 
an actual `destroy` function. And RAII didn't happen with classes.

Mathis noted that his case was a special case. He had this magic 
rebindable thing going on where he had to take manual control of 
lifetime. He supposed it was something that didn't come up often. 
Walter said when you were manually doing lifetime as a special 
case, then it was a special case and you had to put an `if` in 
there.

I noted that `Object.finalize` in Java had been deprecated 
because it was so easy to misuse. I remembered that Sun used to 
always tell people not to do this or that in `finalize`, because 
it wasn't a destructor, yet people would still do this and that. 
We had kind of the opposite problem in D in that our destructors 
were also finalizers.

Walter said you were never going to get fire and water to mix. 
Reference types and value types were just intractably different 
from each other.

Mathis said this had come up because he had to add that 
rebindable alternative to support `immutable` in `std.algorithm`, 
[which all went back to his DConf '22 
talk](https://youtu.be/eGX_fxlig8I).

He then wrapped up by reporting that Funkwerk were on DMD 2.108 
and LDC 1.38 everywhere and it was working pretty well. Their 
code was internally managed with no major external dependencies. 
They were happy to be off of vibe.d and no longer dealing with 
its issues from compiler updates. And they were happy with 2.108.

### Luís

Luís said he had nothing new to bring, but he wanted to 
understand the status of the AST refactoring project. Weka still 
had linkage issues resulting from the incorrect inference of 
templates and attributes. He thought it was mostly because the 
way we queried the AST was kind of a mess. If we had a way to 
make that better, then it would make other things better.

I said that Razvan was the one who knew all about that. I knew 
that it was mostly Razvan working on it right now, though others, 
including Bastiaan, had contributed. It was also an officially 
sanctioned idea for a SAOC project. So it was in progress, and 
Luís should ping Razvan if he wanted an accurate picture.

Luís closed by saying he'd seen some good features come into the 
language lately that he'd been waiting a long time to have, like 
string interpolation and named arguments. He thought that was 
really cool to see.

(UPDATE: One of the SAOC 2024 participants is working on the AST 
refactoring project right now, and Razvan intends to enlist some 
of his students at the [Politechnica University of 
Bucharest](https://upb.ro/en/) to contribute. He could always use 
more help on this. It's not difficult work, there's just a lot of 
it. But it can be done in small chunks which makes it really 
amenable to contributions from multiple people. This is a 
high-impact project. If you'd like to find out how to contribute, 
please read [Razvan's call to arms on the D 
blog](https://dlang.org/blog/2024/02/22/dmd-compiler-as-a-library-a-call-to-arms/).)

### Mario

Mario had no programming issues to report but said that three 
Funkwerk staff would be attending DConf this year.

Mathis said that Luís mentioning named arguments had just 
reminded him that Funkwerk wanted to use them, but currently 
couldn't because they weren't supported in dfmt. They used to 
allow one of their employees some time to spend on dfmt, but that 
employee was no longer working for them and they were now in a 
gap. He wasn't sure what to do about that. Maybe he could take a 
few days to look at it, but they used dfmt in all their projects, 
so that made named arguments a non-starter for them.

Dennis said he had started using dfmt recently for his own code 
and noticed a few bugs. He'd opened some issues, but it seemed no 
one was actively monitoring it. He said he might take some time 
to figure out how it worked so he could fix these issues. It was 
really annoying when the formatter squashed things in attributes, 
got confused by the colon in named arguments, and completely 
ruined the argument list.

Mario said that he hadn't assigned Mathis to fix dfmt because he 
wasn't sure if it would be a waste of time and money. He knew 
that there was also sdfmt from SDC, and he also knew that 
something had been done with dfmt with regards to integrating 
DMD-as-a-library, but he didn't know what the progress was. He 
didn't want to task Mathis to fix dfmt if the fixes would become 
irrelevant when it was replaced with something else.

Dennis said one of Razvan's students had worked on the project to 
replace dfmt's use of libdparse with DMD-as-a-library, but he 
didn't think it was enabled by default. The last he'd heard there 
was just a switch to enable it and it was still kind of early. He 
didn't think the change would affect the formatting logic, but 
we'd have to see.

I noted that in the past we had discussed shipping a formatter 
with the compiler. We had agreed it would be dfmt because it was 
going to use DMD-as-a-library, which made sense if we were going 
to ship it with DMD. As such, any fixes to the formatting logic 
shouldn't be wasted time.

Luís said that Weka hadn't been able to test the dfmt changes yet 
because they were a few compiler releases behind, but he hoped 
they'd be able to soon. He'd report any issues they found with it.

### Johan

Johan said there were probably a few issues from Weka to talk 
about, but he hadn't collected them before the meeting, so he'd 
bring them up next time.

He did have some news for us. He said that Weka had been 
interested in linking executables with musl libc. He'd fixed the 
last parts that were required to make it happen. He'd upstreamed 
almost all of it to DRuntime. He said that was a nice success 
story. You could now create statically linked executables that 
didn't depend on any libc implementation, and that had been the 
goal.

Next, he requested that we include the language spec in the 
compiler repository to prevent the kind of parallel pull requests 
where you ended up having to revert one if you had to revert the 
other. He was sure it had been debated before, but he would like 
to see it.

Robert said it would be even more awesome if the spec were unit 
tests, i.e., docs that were actually testable. That would 
probably be more work, but it would be really cool. Even with the 
spec in the compiler repository, you'd still have to remember to 
update it. But if you had a spec that failed to compile and the 
documentation was generated from that, it would be a bigger win.

Walter said that was a great idea.

Dennis noted that examples in the spec were currently run if they 
had the proper macro. He said that Nick had been working on 
transforming raw, untested code into runnable examples.

Robert said that was awesome and asked if it included other test 
cases for the language. Sometimes when he read the spec and 
something wasn't clear, he'd go to the tests and see what 
actually came out at the end. If they were included, that would 
tighten up the spec quite considerably.

Dennis asked if he meant linking the DMD test suite with the 
specification, and Robert said yes.

Walter said the test suite was really not meant for public 
consumption. It was not designed to be that way. It was designed 
to make the compiler easy to debug.

Robert agreed but reiterated that sometimes when he read the 
spec, there was some nuance missing, or the wording was such that 
he misunderstood. Reading some obscure source code that might be 
hitting his specific need helped him in that regard. He 
understood where Walter was coming from, but it was just an idea 
to float.

Walter said Robert was right, but he pointed out that every time 
he read a computer software specification, including language 
specifications, the specs on how the function call ABI worked 
always confused him. So he would write little test cases, compile 
them, and see what happened. Robert said he was a lot lazier than 
Walter and was looking for somebody who had already written the 
thing he was interested in.

Walter said that once he had written the code, compiled it, and 
seen what happened, then he'd go back and read the spec again and 
it would make sense. He wished he knew a way to make a better 
correspondence between the specification and understanding it.

Luís said his experience in relation to Walter's had been the 
opposite. For example, reading the D spec, assuming the ABI 
worked as described, and then experimenting with DMD and finding 
the ABI didn't work as specified. For example, arguments were 
getting reversed. In the calling convention, the arguments were 
supposed to be one way, but in the compiler, they were the other.

He said we had two different kinds of tests on the compiler. We 
had all sorts of weird tests for specific compiler bugs. He 
thought Robert was describing tests that served as examples but 
that also confirmed the spec was conformant with what we wanted 
to say.

Walter said if the compiler and the spec didn't match, that was a 
bug and should be reported. They happened, but if they were 
reported we could knock them down one by one. Luís said he had 
tried to fix the calling convention bug, but it was a very hard 
problem because it was everywhere in the DMD backend.

Walter agreed that the calling convention was a large source of 
complexity, especially in the backend because it supported 
multiple calling conventions. It was a large tangle of code.

Luís said one of the problems he saw was with DMD's math 
intrinsics. They were implemented based on the calling 
convention, but because the argument order was actually reversed, 
we now needed to change all of those intrinsics. He'd submitted 
some PRs to try to fix them, but the tests failed. From talking 
to Iain, he thought GDC was the most ABI-compliant D compiler.

Johan brought us back to his original point: if we had the 
compiler and the spec in the same repository, it would be easier 
to know which language spec matched the version of the compiler 
you had.

As for testing the spec, he said there were companies out there 
that made test suites for C. This was tough work. He thought it 
was a completely separate issue from having the spec and the 
compiler in the same repository. It would be a huge amount of 
work to write all the lines of test code that followed from a 
single line of the spec. It would be great, but it was a huge 
project for sure.

As for the ABI, he felt that was outside of the language spec. To 
him, the spec was at a higher level. We implemented a C ABI 
because we had to, but otherwise, the ABI was quite open. That 
was on a lower level. The more interesting thing for programmers 
was on the higher spec level, and that was linked with your 
specific compiler version. When they were in separate 
repositories, then you had to figure out which dates matched 
which and it got quite tricky quite quickly.

Walter agreed that the language spec and the ABI spec were two 
distinct things. He noted that after the C language was 
standardized, a company made a test suite out of every piece of 
code in the C spec and then released it. Every C compiler fell 
flat on its face. It took years for all the C compilers to 
successfully run the tests extracted from the spec. This was the 
whole problem.

Mathis suggested that when you're looking at the language spec 
and something doesn't make sense to you, but then looking at a 
test causes it to make sense, then that was a test that should go 
into the spec. You didn't have to put it in at that moment, but 
you could file an issue for it. Walter agreed.

Luís gave an example of a situation Weka had encountered where 
having the spec and the compiler in the same repository would 
have been a help.

Walter said he had no objections to it. He didn't think it would 
be a big problem to merge them.

Finally, Johan reported that Weka were still on LDC 1.30, but 
they were close to upgrading to 1.38, which was D 2.108. He had 
already seen some changes in 2.109 to which he thought it would 
take them some time to adjust, so the plan was to upgrade first 
to 2.108 and continue from there.

Luís said he was very excited to use some DIP 1000 features. The 
DIP 1000 changes were what had been keeping them on an older 
version because of breakage, but that had been fixed.

### Mathias Lang

Mathias wanted to reiterate the importance of tooling, and he 
thanked Dennis in advance if he would take a look at dfmt. 
Symmetry were using a formatter as well, though they were using 
sdfmt since Amaury maintained it. But a lot more could be done 
with the tooling and he thought it would integrate with their 
projects quite well. DMD-as-a-library would enable a lot of good 
things, and that would be enabled by the AST refactoring project. 
If we had to choose between an ARM backend for DMD or an 
immutable AST, his vote would go for the immutable AST.

### Dennis

Dennis had been working with COM lately. It had a special calling 
convention with named arguments, and D had named arguments, so he 
thought it would be nice to try to combine them. But `opDispatch` 
and `opCall` didn't support named arguments, and the DIP didn't 
provide any mechanism for it.

He had an idea to add something like `opNamedDispatch` that would 
push the named arguments as a template argument as a string 
array. That would make it work. Walter thought that sounded fun.

### Walter

Walter said that he'd been posting updates to X about his daily 
progress on the AAarch64 code generator. He was currently working 
on the function call return sequence. Having spent some time with 
it, he had slowly begun to understand the ARM instruction set. 
Some parts of it were still baffling. For a supposedly simple 
instruction set, it was extremely complicated. He had to write 
little snippets of code and try them out on godbolt to see what 
they actually did.

He'd gotten to the point where he thought it was going to work, 
but getting the details done would take a bit of time. He 
expected to have the basics running fairly soon.

Johan asked if Walter had a machine to run it on. Walter said 
he'd just been using godbolt. He'd write a function that was two 
lines or so and wanted to see what instructions were emitted for 
it. How were they doing pushes and pops? How were they doing 
function calls? Godbolt was perfect for that.

He said he also had a Raspberry Pi. He expected that Raspberry Pi 
users would be a potential user base for a simple AArch64 DMD 
backend. That's what he planned to use as a target machine. The 
code the compiler currently emitted didn't handle fixups in the 
elf object files, so actually trying to run the generated code 
wouldn't work.

Unfortunately, the elf generator code was a rat's nest. He'd been 
refactoring it to make it more tractable. With the refactoring, 
it should be easier to implement the AArch64 fixups, which were 
completely different from the x86 fixups.

He'd already written an object file dumper and a disassembler for 
it, so the pieces were slowly coming together. Once he had it 
working on the Raspberry Pi, he'd get a recent Mac with an ARM 
chip in it and make it work there.

Johan said he'd thought Walter was a Windows user. He'd learned 
there were some good Windows ARM laptops these days. But at some 
point, it would be good if Walter had a machine to run it on.

Walter said the Windows backend would probably be the last step, 
primarily because he found the Windows ABI poorly documented and 
much more high-effort to get things working. He also didn't like 
working with the Microsoft SDK. It was excessively massive and 
complicated, and it was hard to find anything in it. It had too 
many directories and files, and he just didn't like messing 
around with it. That was also the reason Windows support was the 
last to get implemented for ImportC.

He said the Linux development tools were just so much better. 
They were easier to understand and oriented toward a command 
line, not a GUI. He found them much simpler to develop code on 
and much easier to understand what was happening. So he was going 
to defer the Microsoft problems to last.

Martin noted that DRuntime most likely needed work for Windows 
AArch64. Johan said that, indeed, LDC probably would not run on a 
Windows ARM laptop. It would be nice if we had some developers 
that had one to work on it. It wouldn't necessarily have to be 
Walter. Maybe one of the core contributors, or someone with skin 
in the game and the motivation to fix the final bits and pieces.

Dennis asked what Walter planned to do with the PR. Was he going 
to keep adding to it and merge the whole thing once the test 
suite passed, or was he going to merge a minimal working version 
and build on it?

Walter was aware that it was getting larger and larger, and there 
was no end to that in sight. He was constantly rebasing it so it 
didn't become hopelessly divergent. It was so large now that it 
was frankly impossible to review. He asked if Dennis thought it 
reasonable to simply mainline it even if it wasn't a fully 
functional backend yet. That would make it easier to manage and 
avoid it becoming impossible to merge while making future 
additions easier to review.

Dennis thought in any case it would be nice to have the flag and 
the skeleton that the actual code generation could build off of. 
Walter said in that case, he'd be happy to merge it if it passed 
the test suite. Dennis asked if there were any ARM tests in the 
test suite that actually ran the generated code. Walter said 
there were not.

Dennis asked if it was just testing against expected assembly, 
and Walter said that's what he was doing. He was doing visual 
checks of the assembly. The `-vasm` switch had turned out to be 
really useful for that. He compiled with `-vasm`, looked at the 
disassembled code, and compared it with what godbolt put out. 
He'd even take what godbolt emitted and put it into the 
disassembler as part of the test suite.

Dennis said he could take a look at it some time. He wanted to 
look at the code, but it wasn't something he'd be able to do in a 
single evening. Walter said that was exactly the problem, so he 
was all for merging it and continuing on from there.

## The Next Meetings
Our October Monthly meeting took place the following Friday, July 
12th. Our next quarterly was on Friday, October 5th.