D Language Foundation August 2023 Monthly Meeting Summary

[max haughton]

On Tuesday, 22 August 2023 at 12:59:12 UTC, Mike Parker wrote:

> ### Robert
> Robert said he was happy. Dennis asked if the compiler was fast 
> enough (as regular readers of these summaries may know, Robert 
> often says the compiler is too slow). Robert said it's still 
> slow. He just wanted to see LSP support in the compiler and 
> take all the good things.
>
> Walter noted he'd been working on improving compilation speeds. 
> He'd realized that the compiler was being compiled with 
> exceptions, so he'd been working on making it compile without 
> exceptions to see what effect that had on speed. It might be a 
> small difference, but every little difference helps.

Follow the memory. I don't think the exceptions will make any 
difference at all. The compiler allocates soooooo much 
unnecessary memory and lays out what it does use very badly. 
This, and doing less work, is to way to make an algorithm whose 
complexity you can't change faster. I have recently started using 
a tool called bytehound which can show you some nice metrics from 
allocations via LD_PRELOAD-ing a magic .so rather than emulation 
or recompilation, we should generate (I will do this during dconf 
perhaps) reports at CI time to find hotspots where possible.

Using some kind of copy-on-write would be interesting because the 
compiler preemptively allocates a lot during semantic analysis.

Also please check that using the new -nothrow doesn't break the 
ICE error messages and so on when its enabled for the compiler.

> Adam said his library compiled in half a second. Robert said 
> that was still too slow. His work project didn't compile in 
> half a second. He said his goal was that by the time of the 
> key-up event after pressing 'Enter' on Ninja or whatever build 
> tool/compiler he's using, it should have already restarted the 
> application. Átila agreed that half a second was too long.
>
> Adam brought up D1. An old game he wrote back in 2007 compiles 
> and links in 10 milliseconds. If you looked at a big chunk of 
> the difference between D1 and D2, `object.d` back then was 
> around 600 lines. Now it was much bigger, and that adds up. 
> Walter said that he was a bit upset with the state of 
> `object.d`. It was supposed to be a minimal thing and had 
> turned into a kind of garbage dump somewhere along the way.

A lot of this time is actually just linking druntime (approx 2/3 
of the ~.3s on my machine), for the record, rather than literally 
object.d being compiled.

> ### Steve
> Steve said that Dennis had been doing a fantastic job with his 
> PR on the compiler part of static associative array 
> initialization. Steve thought it was about ready to go if 
> someone could push the button. He didn't want to be the one to 
> push it, because he didn't know the compiler code very well and 
> couldn't vouch for its correctness. ([The PR is still 
> open](https://github.com/dlang/dmd/pull/15468) as I write.)
>
> ### Dennis
> Dennis said he'd been working on that AA PR, but had nothing 
> else to report. He still had some other things to pursue, like 
> looking at pattern support for the deprecation (-d/-de/-dw) and 
> obsolete warning (-wo) switches, as well as the `standalone` 
> attribute or pragma for static constructors.
>
> ([In our July 
> monthly](https://forum.dlang.org/thread/amsduzcnbgvptaxjtlcn@forum.dlang.org), Adam proposed a means to annotate static constructors that could make them independent of cycle detection. [In a subsequent planning session](https://forum.dlang.org/thread/vccxtwaaxqxxjojgxbhj@forum.dlang.org), we agreed to implement it. Since the August meeting, [Dennis has submitted a PR](https://github.com/dlang/dmd/pull/15537) for the `@standalone` attribute, but in the PR discussion thread, Walter thinks it should be a pragma.)
>
> ### Martin
> Martin said the bump to version 2.104 of the frontend went well 
> on the LDC side and was quite painless.
>
> Next, he had an interesting side note inspired by [a forum post 
> he'd 
> seen](https://forum.dlang.org/thread/ofbvqasfvxylbxxpwhja@forum.dlang.org) about getting LDC to work on a 8-bit AVR platform. He'd looked into it and found it an interesting question in terms of whether and how we want to support architectures with pointer sizes less than 32 bits. Currently, the frontend has a hard-coded limit: `size_t` and `ptrdiff_t` are either 32 or 64 bits, nothing lower than that. It's a frontend check, and the definitions in DRuntime directly depend on the compiler's internal type because they use the `typeof` expression.
>
> He had tried to make the frontend accept 16-bit and 8-bit 
> `size_t` and `ptrdiff_t`, but doing that bumps up against the 
> integer promotion rules. The promotion rules can work in some 
> specific cases when using a smaller `int`, but when you have 
> something like the length of a slice and do a little plus one 
> or minus one, that's a problem. You need to cast it. So either 
> we need to change dealing with the promotion rules in this 
> regard, try to mimic C++ in this regard, or hopefully do 
> something better. He said Adam had proposed something in a past 
> discussion that would handle it generically so it's still 
> target independent.
>
> Walter wondered if Rust supported less than 32-bit compilation 
> and what compromise they made for that. Martin said he wasn't 
> sure. He'd only checked C and C++. Robert looked it up and 
> found that Rust has a tiny bit of support for 16-bit MSP340, 
> but it's a big asterisk and everything else is 32 or 64.
>
> Martin wasn't sure if it was even worth looking into these 
> architectures, because he had no idea how long they'd be 
> around. For low power consumption, even if you're playing a few 
> years ahead, he thinks it's going to be 32-bits even for those 
> very small and power-efficient sensors, or whatever. But he 
> couldn't look into the future, so he had no real idea.

There are some 8 and 16 bit parts still floating around (ultra 
cheap will probably never go 32) but its definitely a trend.

> ### Ali
> Ali had mentioned [in our July quarterly 
> meeting](https://forum.dlang.org/thread/jzlympfqmwckaiuhqoiz@forum.dlang.org) that his company (Mercedes Benz Research & Development North America) was looking for a D programming intern. He'd since contacted Mike Shah at Northeastern University. As an example of industry/academia collaboration, Mike had recommended a strong candidate, and the company had already hired him. They planned to bring on another intern who comes with a good recommendation. So the D team is increasing to three people.

Very nice to hear.


> Walter said he'd thought about Robert's observations that until 
> we have the compiler as a language server or a library, we're 
> going to be in trouble. So he'd been going through the compiler 
> frontend and fixing it so that it's much more amenable to 
> having DMD as a daemon.

Amenable yes, but this is all very easy to work around if the 
compiler itself was fundamentally suitable to working as a 
library.

> He'd made quite a number of changes for that. Some of the PRs 
> were still sitting there, waiting to be pulled. The idea was to 
> look at the subsystems of D and remove their dependencies on 
> global state, and dependencies on the environment and the 
> operating system, so that they have a simple entry point where 
> one function takes all the parameters it needs, does its 
> operation, and returns data. Then the caller can decide whether 
> to write a file or what to do with error messages, and so on.

> He said that work was kind of satisfying when he got a piece 
> set up like that. The error sink had been his first attempt in 
> that direction. The lexer and parser were now capable of being 
> divorced from the rest of the compiler and are restartable. It 
> also means the LSP daemon or whatever will be able to be 
> multi-threaded, because you can be doing multiple compiles at 
> the same time with this setup. He's still got a ways to go, but 
> it's proceeding.

I think you could already do this, but mostly good changes 
nonetheless. The lexer I think is actually not safe to run in 
parallel because of the timestamps, but I think Stefan had a PR 
to fix that a while ago that got stalled for xyz reasons.

> He was also looking at redefining the interface to the glue 
> code, so that should make Iain's and Martin's lives a little 
> easier. Martin said he wasn't so sure. He thought the interface 
> in its current state was absolutely fine.

Its not great.

> He saw this as a sort of existential thing for D. He had come 
> around to Robert's point of view and felt it was important to 
> get this moving. Razvan wasn't in the meeting, but Walter was 
> curious about his thoughts about all of this since he'd been 
> working on DMD as a library. He asked Robert if the work needed 
> for both DMD as a library and the LSP server was the same.
>
> Robert said DMD as a library is a big building block to build 
> the LSP, but there was more to it in terms of building a server 
> that takes commands, runs the compiler, and returns the 
> information as JSON. The compiler library is a big part of 
> that. His fingers were crossed that the rest was just glue. He 
> thanked Walter for working on this. Walter said Robert should 
> be thanked for making the observation that gave him the impetus 
> to get this done and realize its importance.

The compiler needs to be able to reuse its previous work to have 
any chance of success here. If you have a large codebase its not 
really practical to have two dmds running at the same time after 
you save your file. This is more achievable than you might think 
but I think it really needs to be contemplated what these tools 
actually do rather than just some vague of dmd-as-a-library + 
some magic - its not just an exercise to the reader.

Also ergonomically you still want to be using the same "pipeline" 
the regular compiler is using rather than gluing the bits 
together yourself when you use the library, there will always be 
minute details. I think the plugin system ldc is heading towards 
(upstream this or things will fragment!) is probably uglier but 
more tractable than building discrete tools using dmd as a 
library.

> Then Walter thanked Steve and Dennis for working on the static 
> AA initialization. For a long time, he'd thought it was 
> completely impractical. Steve had convinced him it could be 
> done reasonably. It was a nice improvement that didn't need 
> anything different because it was just turning on something 
> that hadn't worked.
>
> Steve said there was a note in the spec saying this was an 
> intended feature that wasn't implemented, so we could finally 
> remove that. He said that Dennis was doing a great job. He 
> sometimes felt that every time he brings up something that's 
> wrong, Dennis has a PR to fix it a day later.

Should be merged - then use `newaa` to implement the runtime AAs 
too, two implementations is horrible.