D Language Foundation April 2023 Quarterly Meeting Summary

[Mike Parker]

The D Language Foundation's second quarterly meeting for 2023 
took place on April 7 at 14:00 UTC. It lasted just over an hour. 
This was the first "standalone" quarterly meeting, where the 
foundation's monthly discussion was split into a separate meeting 
the following week.

Two new members joined us this time, and we look forward to 
hearing more from them at future meetings.

I was in transit at the time the meeting took place, so I want to 
thank Razvan for conducting it in my stead and Dennis for 
recording it so that I can write up this summary.

## The attendees

* Mathis Beer (Funkwerk)
* Walter Bright (DLF)
* Andrea Fontana (2night)
* Mathias Lang (DLF/Symmetry)
* Martin Kinkelin (DLF/Symmetry/LDC)
* Dennis Korpel (DLF)
* Mario Kröplin (Funkwerk)
* Vijay Nayar (Funnel-Labs.io)
* Átila Neves (DLF/Symmetry)
* Razvan Nitu (DLF)
* Robert Toth (Ucora)
* Bastiaan Veelo (SARC)

## The summary

### Mario
Mario started by asking about the future of formatters in D. 
Funkwerk has put a lot of [effort into 
dfmt](https://github.com/dlang-community/dfmt), and they run it 
over all of their code, but pull requests for it have long 
waiting times and new releases are infrequent. Typically, when 
the Funkwerk programmer who contributes to dfmt makes a PR, it 
goes for weeks with no progress. There's also sdfmt (part of [the 
Snazzy D Compiler project](https://github.com/snazzy-d/SDC) 
maintained by Amaury Séchet) that seems to get more use. The 
question he was looking to answer: should Funkwerk keep putting 
effort into dfmt, or switch to sdfmt?

dfmt was created and originally maintained by Brian Schott, but 
for some time now has been under the dlang-community group at 
GitHub, and no one at the meeting could say who was responsible 
for it now. Mathis Beer noted that Jan Jurzitza had mostly been 
the one merging things.

This was followed by a 15-minute discussion that went off on a 
couple of related tangents, but ultimately came down to two 
conclusions:

* Funkwerk should get increased access to the dfmt repository
* Razvan will have a new crop of students next fall, and he can 
find one to work on integrating dmd as a library in dfmt

The intervening discussion flowed as follows.

Mathias Lang said that Symmetry uses sdfmt since the maintainer 
is also part of Symmetry's team. The perceived advantage it has 
over dfmt is that it's based on the compiler as a library. He 
posed two questions: do we want to have one formatter we endorse 
as the D Foundation and, if so, which one should it be? If we do 
endorse one, then we should distribute it with the compiler. 
Walter agreed that a formatter should be part of the compiler 
release.

Mathias added that it would be much better if we have the 
compiler as a library. Razvan agreed and said that if we could 
convert dfmt to use dmd as a library [rather than 
libdparse](https://github.com/dlang-community/libdparse), that 
would be a step forward. Dennis questioned that point. The 
formatter only needs the parser. libdparse takes everything into 
account for syntax handling, but the dmd parser skips over some 
things (such as comments and unit tests). And even if the dmd 
parser is changed to account for more, how is it a better option 
than libdparse? This prompted a brief side discussion that 
resulted in general agreement that the main problem with a 
third-party parser is that it falls behind (some recent examples 
were cited) and will rarely be in sync with the latest compiler 
release. Átila added that even when it is current with the latest 
language features, the fact that it's a separate implementation 
means there are bound to be differences that manifest as bugs.

Razvan said that another issue with libdparse is anything using 
it that needs semantics, [like 
dsymbol](https://github.com/dlang-community/dsymbol), has to come 
up with something mimicking the compiler's behavior. He has a 
student currently working on replacing the use of libdparse and 
dsymbol [in 
D-Scanner](https://github.com/dlang-community/D-Scanner) (see 
Lucian Danescu's DConf '22 talk, [Integrating the D Compiler as a 
Library in D-Scanner](https://youtu.be/JYkb3PjIn4c).

Walter noted that he had spent some time converting dmd's lexer 
and parser into standalone components. At the time of the 
meeting, the lexer was fully independent and the parser was 
mostly independent. There was still some work to be done to 
remove the parser's dependencies. He said he could get back on 
that and then it would be better suited to replace libdparse. 
That won't solve the semantic issues, but at least then the lexer 
and parser could be the same as what the compiler uses. ([He 
posted in the 
forums](https://forum.dlang.org/thread/u0vqo6$1bc6$1@digitalmars.com) a couple of days later that the lexer, parser, and ImportC parser were now all independent.)

Walter then suggested that, down the road, we would ideally have 
only one formatter. We can continue to update dfmt, but if sdfmt 
is getting more use, then maybe it's the better candidate to be 
the sole formatter in the future. Mathias noted that sdfmt is 
implemented with the SDC frontend, and since that's a parser 
completely independent from the D frontend used by DMD, LDC, and 
GDC, then sdfmt has the same problem as dfmt does using libdparse.

This was followed by some down-in-the-weeds discussion about 
specific semantic problems and how to adapt DMD as a library 
before coming back to Mario's original question and the 
conclusions I cited above. (One of Funkwerk's employees has since 
been granted increased privileges for the dfmt repository.)

### Mathis Beer
__libRebindable in Phobos__

Mathis said he was working on PRs to push libRebindable into 
Phobos ([see his DConf '22 presentation for more on 
libRebindable](https://youtu.be/eGX_fxlig8I)). He's thrown out 
the parts that are useful only in corner cases and is keeping the 
parts that are independently useful in the general case. He was 
aiming for a version of libRedindable into which you can push an 
arbitrary data type, pull it out again, and have all the 
reference counting and so on accounted for correctly.

He's noticed by pushing more immutable data types that something 
like half of Phobos is unprepared for it. For example, 
`maxElements` seems like a simple function, but its current 
implementation cannot handle immutable data types. He had a PR to 
work around it that was getting reasonably scrutinized as 
terrible, but if he could throw libRebindable at it and other 
such issues he's hopeful something can be done about it. So if 
the libRebindable stuff gets merged, things should be looking 
better.

__Profiling compiler performance__

He next brought up the persistent issue of slower compile times. 
He noted that LDC now has a "very, very snazzy" command-line 
option, `-ftime-trace`, that outputs profiles of LDC compile 
times in the Chrome Trace Event format, which can then be seen as 
a flame graph in a viewer. It's amazing and he loves it. Running 
it on Funkwerk's internal codebase, he found that one module 
which generates enums for HTTP error codes (with a CTFE loop and 
a bit of string processing) takes 300ms of compile time on its 
own. If someone is looking for a toy to play with to understand 
what the problem is with pretty straightforward CTFE that has a 
lot of hidden costs, this would be a good example. He was 
considering posting it in the forums.

Dennis agreed that it's a very useful tool. He had employed it to 
shave 25% off of the front-end time. He added that Johan Engelen 
had recently enhanced it to track CTFE (previously, it only 
tracked semantics) and also to output to a text format that's 
easier to read in a text editor. Dennis was thinking about making 
a tutorial on how to use it for the foundation's YouTube channel.

__Using Phobos in the compiler__

Finally, out of personal interest and unrelated to anything. 
Phobos had been prohibited from use in the DMD codebase because 
of, as he understood it, the need for C++ bootstrapping. He asked 
if that was a permanent situation, or if there could ever be a 
point where D's presence in GCC is widely spread enough that we 
can rely on that being present and dogfood Phobos more in the 
compiler itself.

Walter said he'd rather not use Phobos in the compiler. If it 
relies on Phobos, it makes the compiler more difficult to build 
and test because of a circular dependency: the compiler relies on 
a working Phobos, and Phobos relies on a working compiler. He 
doesn't think that using Phobos in the compiler would be enough 
of an improvement to outweigh that.

Martin noted that LDC does use Phobos for a few LDC-specific 
parts, but it is kind of a problem. It means you have to be 
compatible with all Phobos versions from the oldest version of 
the compiler that can be used to compile the compiler up to the 
current release. He agreed with Walter that we should minimize 
the use of Phobos.

Walter added that we've had problems now and then where we can't 
test the compiler because it fails somewhere in compiling 
DRuntime, and then you can't build the compiler to test it and 
find out why it's failing. That's a large time sink to resolve 
every time it crops up. Trying to add Phobos into that would be a 
large nuisance. The compiler should have minimal dependencies so 
that they don't get in the way of testing it.

### Robert
All the talk about code formatters brought to mind something 
Robert hadn't intended to bring up, but it's something he'd 
wondered about before. He asked if we had put any thought into 
automatically upgrading old code after deprecations and such on 
new compiler releases. It's no problem when you only need to 
update a function or two, but when a deprecated feature permeates 
your codebase, fixing it is tedious.

Mathias Lang said it had come up in the past and mentioned Go's 
`gofix` as an oft-cited example, but he wasn't aware of any work 
that had been done on that front.

Dennis said this depends on what the deprecation does. He brought 
up the deprecation of `alias this` in classes as an example that 
has no trivial fix to make everything work again. It may require 
restructuring code. There was a brief bit of discussion of what 
could be done in that specific case, but Razvan quickly brought 
things back on topic. He said that we could surely implement some 
automatic fixes for trivial cases. Atila said "yes and no". 
Because of templates and mixins, it's not always possible.

Bastiaan [linked dfix](https://github.com/dlang-community/dfix) 
in the chat. It hadn't been updated in years, but it's the sort 
of thing Robert asked about.

Razvan suggested that if we move forward with integrating dfmt 
into the compiler releases, then anytime someone implements a 
deprecation, they should also implement a compiler switch for the 
formatter to update code when it's possible. That seems like the 
best solution.

Next, Robert said he had begun working more with dub. He noticed 
there once was a bounty for modifying dub so that it detected 
file changes via hashes rather than timestamps. He asked if that 
had been implemented. Mathias Lang said we're still using 
timestamps. (He was referencing a bounty that Symmetry had posted 
for this. I know at one point they had selected someone to work 
on it, but as far as I know, the bounty is still open at $2,000. 
We no longer have our Flipcause account where we were posting 
bounties, but I saved all the relevant details for each of the 
open ones before our account was closed. I'll post them again 
once we decide on a medium for them.)


### Bastiaan
Bastiaan said he'd been maintaining [the Pegged 
project](https://github.com/PhilippeSigaud/Pegged) for the last 
few years. Phillipe Sigaud, the repository's owner, granted him a 
maintainer role for the repository a while back. He thinks it's a 
good candidate for addition to the dlang-community organization, 
and that's something we should look into making happen. Phillipe 
is willing to hand it over.

Mathias Lang said that if Phillipe could invite him to the 
project with full privileges, then he could transfer it to 
dlang-community. He'd recently done the same thing to [transfer 
ddox to dlang](https://github.com/dlang/ddox).

Next, Bastiaan said he's been a frequent user of the Windows 
installer. There's a regular issue with it that when the 
installer is finished, there's nothing there. Trying it once more 
works. He thinks it's worth a bug report, but he doesn't know 
where to look for the cause.

Finally, he's noticed with debug builds on Windows that sometimes 
the linker complains that the debugging information is corrupt 
and he should recompile, but then after a recompile, it persists. 
He found that removing several instances of `__FILE__` and 
`__LINE__` causes it to go away. He didn't know if that was part 
of the problem or not. He noted that dub still has the problem 
that file paths can sometimes be very long, and suggested this 
issue might be related to that. The dub problem had been 
addressed in a recent PR, but not yet released.

There was some discussion about whether it was reproducible. 
Dennis suggested the next release of dub should solve the 
problem, and to report it if it doesn't. Martin agreed and 
reminded everyone that Symmetry had encountered that issue 
frequently. He wasn't sure if the fix will solve all cases, but 
it should solve most of them. He talked about a workaround 
involving the Windows registry and a manifest file for the 
executable to enable the C API functions to handle long paths. If 
the prebuilt dub had the manifest, then users could do the 
registry hack themselves and *all* of these problems would be 
gone.

Mathias Lang said the reason that the bug started triggering was 
because of a fix for another issue. Before, dub was using 
relative paths in quite a few places, all relative to the current 
directory where things were being built. That caused issues with 
system-wide packages. For example, you couldn't install a package 
in `/usr/lib`, as doing so would cause dub to try to write there. 
He fixed that by adding a central build cache next to the package 
cache so that all of the paths are absolute. That has resulted in 
longer file paths.

### Andrea
Andrea introduced his company, 2night, based in Venice, Italy. 
They create interactive experiences for bigger international and 
Italian brands. They usually work with Raspberry Pi and other 
embedded platforms and with a lot of different technologies on 
the software side. In the past year, 2night had increased their 
use of the D language. They had two main issues.

The first was that D support for 32-bit Raspberry Pi and other 
ARM platforms (via LDC) was a couple of years out of date. Martin 
said that currently, LDC only supports 64-bit ARM and supports it 
well, and that includes all of the newer Raspberry Pi releases. 
He's not interested in maintaining support for older 32-bit ARM. 
He was going to mention that someone had ported a recent build of 
LDC to 32-bit ARM, but as he was saying it he realized it was 
Andrea who had done so. Andrea said they'd needed it for a 
project in Venice. He'd managed to get it working but hadn't 
tested it deeply.

Martin thanked him for his contribution. He said it didn't work 
in 100% of cases, but 32-bit ARM support wasn't CI tested anyway. 
And there were unit tests that had never worked on 32-bit ARM. 
64-bit ARM is much better, much more mature, and is CI tested. 
Moreover, the 64-bit ARM ecosystem is more homogeneous than that 
of 32-bit ARM, where there are many ABI differences and other 
ugly problems. That's why he's only focusing on 64-bit. Andrea 
said he would look into switching to 64-bit.

The second issue was the integration of ImportC in the D 
ecosystem. They use ImportC a lot, but it doesn't work well with 
dub and other tools. It was a bit rough to use ImportC inside a 
project, with many steps that were difficult to automate. Walter 
asked him to write up a more detailed account of their issues 
with ImportC and post it to Bugzilla. Andrea agreed to do so. 
Dennis asked if Andrea had added the `.c` files to dub's source 
list, and Andrea said the last time he'd tried it didn't work.

Martin noted that you don't get all of the latest ImportC 
features with LDC. One important missing feature is the implicit 
invocation of the C preprocessor which has been part of DMD for a 
couple of releases. Using DMD might be better in that regard. 
Andrea said they're using that with a different project on an 
Intel platform, and the last time they'd used it was last summer. 
He doesn't recall that implicit preprocessor invocation was 
implemented at that point. He was only able to get it to work 
with [the Tiny C Compiler (TCC)](https://bellard.org/tcc/). He 
would have to try again to see what has changed, then open bug 
reports on anything he finds wrong. Walter asked him to report 
even small issues, as he wants to fix them all.

### Vijay
Vijay started by saying he's very happy with the D language and 
the build tools. He develops with emacs in a Linux environment, 
and he's found that the language and tooling have been very good.

He gave some background on his past work helping businesses where 
he'd worked find massive savings on their cloud computational 
costs by making their services more efficient (his DConf '22 
talk, [Efficient Microservices using 
D](https://youtu.be/KXdWQhPNg2c), was related to this). He 
thought he'd be able to build a business out of that with D, but 
after a year, he'd learned that investors tend to prefer cutting 
costs by laying off employees first, so the demand isn't there.

The second thing he'd observed is that D has very low library 
coverage, so he'd been porting over several libraries from 
Python, Java, and C++. It's easy to do for anyone familiar with 
those languages, and even individuals can move quickly to get 
things done that most teams would struggle with. However, he sees 
this as a symptom of a bigger problem: the D community is quite 
small, so the number of people who encounter random problems and 
solve them is also small. This has a feedback effect in making it 
harder for people to adopt D because the libraries they need are 
missing. He doesn't have an answer for how to solve it.

Walter said the point of ImportC was to expand D's usability with 
existing libraries. Vijay said that one of the things younger 
developers rely on when using libraries is popup documentation in 
their IDEs. While ImportC provides binary compatibility, it 
doesn't make documentation or autocompletion available through, 
e.g., the Language Server Protocol (LSP). The ability for ImportC 
to do that in combination with autocompletion might further 
accelerate integration, though he knows that's separate from 
Walter's efforts.

Dennis said he could relate to that. He had recently used ffmpeg 
with D. Instead of using the existing bindings, he tried using it 
with ImportC. The lack of autocompletion was notable, and he sees 
that as something that will hold it back. serve-d gives us all of 
that for D code, but we have nothing for ImportC.

Andrea said he'd had the same problem, also using ffmpeg. This is 
the kind of thing he meant when he said ImportC doesn't integrate 
well with the D ecosystem.

Walter noted that there'd been talk at past meetings about 
getting LSP support into the compiler. He hadn't yet looked into 
it, so he didn't know what it would take to get there. (And for 
the record, Walter was receptive to the idea in those past 
discussions.)

### Everyone else
At this point, Razvan noted that the attendees associated with 
the DLF would normally take their turns now that the industry 
reps had all had a chance to speak. Instead, since we're 
splitting the monthly meetings out of the quarterlies going 
forward, he asked if any of them had anything to say that 
couldn't wait a week. None of them did. He asked for any other 
comments or questions.

Walter took the opportunity to thank all the industry reps for 
joining us and telling us about their experiences. Bastiaan said 
it's very nice that we're listening. It's great to have a place 
to come and discuss issues in person.

Robert was curious about the monthly meetings and wondered if he 
could or should join. Razvan and Dennis described how they 
usually work and noted that I post the summaries here in the 
forums. They said I had mentioned that I'd like to expand them by 
regularly bringing in more contributors, but they weren't sure 
what my plans were. They suggested he ask me about the details. 
(Yes, I do want to expand them, and I did provide more info for 
Robert.)

Vijay said there are two real-time chat channels listed on the D 
homepage (Community Discord and IRC) and wondered which was the 
primary channel. Razvan said that Slack is the "official" place 
where contributors reach out to each other when they need to, but 
it's rather quiet there. Discord is busy enough that he has to 
keep notifications muted. He noted that I've often said the 
Community Discord is flourishing and is the entry point to the D 
community for more people than the forums these days.

Walter said he finds the real-time chat formats disconcerting. He 
prefers newsgroup-style communication. He said that he tries to 
maintain a D presence at Hacker News and welcomes anyone else to 
help out. Click on his name and you can see where the action is. 
He encourages everyone to monitor sites like Hacker News and 
Reddit to participate in conversations about D and initiate posts 
about D when they can.

## The next meeting
Thanks again to Razvan for running the show. He did such a good 
job that I'm tempted to let him keep it.

If your company is using D in production and you'd like a seat at 
the quarterly table, let me know! Now that we've split the 
monthlies out, we've got plenty of room to add more people. We 
also welcome anyone to join us for our monthly meetings if they 
have a specific issue they'd like to discuss with us. Again, let 
me know, and I'll set it up.

The next meeting was a monthly meeting that took place on April 
14 at 14:00 UTC. I'll have the summary for that in a few days.