D Language Foundation February 2023 Monthly Meeting

[Mike Parker]

The February meeting took place on February 3rd, 2023, at 15:00 
UTC. The following people attended:

* Walter Bright
* Iain Buclaw
* Ali Çehreli
* Martin Kinkelin
* Dennis Korpel
* Mathias Lang
* Átila Neves
* Razvan Nitu
* Mike Parker
* Robert Schadek

The meeting lasted just under 50 minutes. We don't normally 
constrain the duration of our meetings, but this time we had to 
keep it under an hour because we had another one scheduled at 
16:00 UTC. We've been having weekly meetings every Friday since 
late January. I'll be able to talk about those at some point, but 
not just yet.

## The summary

### Me
I opened the meeting by letting everyone know that Ikey Doherty 
of the SerpentOS project was scheduled to join us for our next 
meeting. We'll still be having our weekly meetings in March, so 
to make sure Ikey has enough time to address us, we should start 
our March meeting at 14:30.

Later, I talked about videos. The DConf Online 2020 & 2021 videos 
I've rendered have all used the H.264 encoding, as that's what my 
editor supported at the time. I have a different editor now, but 
I still rendered the DConf Online '22 videos in the same 
encoding. I believe videos from the last three DConf editions are 
also encoded in H.264, and I don't know yet about the older ones.

Now that we're storing our videos on Backblaze, I've begun 
rerendering them as H.265 to cut down the file sizes. Our monthly 
Backblaze bill for storing all the videos and the compiler 
downloads archive is less than $3.00, so it's not like we're 
paying a lot. But every penny saved counts.

### Mathias
Mathias had recently been using D for scripting a web interface. 
He's missing the ease of use of having a lot of things in the 
standard library. We have an okay JSON package. That makes a 
world of difference, but if you have to do something with, e.g., 
YAML, you have to take the extra step of searching for a library. 
He thinks we should discuss bringing more things into the 
standard library.

Robert agreed. Specifically, regarding text file formats, he 
thinks we should have a package that supports multiple formats 
with a somewhat standardized API. YAML, SDLang, XML, and JSON 5 
would be a good start.

### Razvan
Razvan informed us that the deadline for GSoC applications was 
coming up. He still needed a list of potential projects and 
potential mentors. [Our projects 
repository](https://github.com/dlang/projects) contains several 
ideas, but not all of them are suitable for GSoC. He suggested 
selecting four or five for the application. In 2021, we submitted 
a link to the project ideas repository. He felt that might have 
harmed our prospects, and thinks they were expecting a distilled 
page of more organized project ideas. He wanted to make sure 
Walter and Átila are okay with whatever projects we add to that 
list.

Robert suggested an email discussion to settle on projects. He 
proposed the text format package as a potential project, and 
Átila gave this thumbs up. The final list of projects Razvan 
submitted is at https://github.com/dlang/projects/pulls. 
Ultimately, our application was rejected.

### Dennis
Dennis reported on his pull request adding named arguments for 
functions. Walter hadn't yet responded to a ping, so Dennis asked 
if he could look at it. Walter subsequently did so, Dennis 
addressed his feedback, and [the PR was 
merged](https://github.com/dlang/dmd/pull/14575).

### Robert
Robert had nothing to report other than his usual complaint that 
the compiler is too slow.

Walter asked if he was using `fullyQualifiedName`. Robert said 
yes, but that's not the problem. It's templates in general. It's 
not that template usage in Phobos is the problem, it's that the 
compiler is too slow in compiling them. Walter noted 
`fullyQualifiedName` triggered a chain of instantiations. [He had 
a PR to replace it](https://github.com/dlang/dmd/pull/14711) with 
a new `__traits`. He suggested Robert see how that impacted his 
compile times. Robert said he had looked at the instantiations in 
his compiles and expected he wouldn't even notice it. That said, 
he thinks `__traits` and `std.traits` are critical for 
metaprogramming and he thinks the PR is a good idea.

Walter doesn't think there's one single solution to speeding up 
template processing. He's implemented several optimizations in 
the past that have sped up things like, e.g., the `AliasSequence` 
template, and he's always looking for other opportunities to 
enhance template processing performance. Robert noted how we had 
discussed the topic [in our December 
meeting](https://forum.dlang.org/thread/uhcopuxrlabibmgrbqpe@forum.dlang.org). He thinks the main issue is separate compilation units. He emphasized that compiling source should no longer be seen as batch processing. It's an interactive game today.

Other than that, he says D is still the best language.

Razvan asked him about the status of the Bugzilla->GitHub 
migration. Robert said he still needed to make time to finish the 
script. Currently, the script has no proper UI. Manipulating it 
requires editing the source. He wants to set up a proper user 
interface so that whoever ends up running it (probably me) can 
have an easier time of it. And he still needed to account for 
corner cases, as when the GitHub API sometimes times out.

I asked Robert to email me as soon as the script is ready. Then I 
can ask Walter to contact Brad Roberts so we can set up a time to 
put the Bugzilla database into read-only mode. Robert suggested 
we don't need to contact Brad. All we need is the GH auth token 
and then everything is handled by the REST API. I argued that if 
Bugzilla is updated during or after the migration, we'll miss the 
new comments or issues. Robert said the script can remind the 
user to check for new issues immediately after the migration, and 
that a comment can be left at the bottom of each existing issue 
indicating that it has been migrated and any future comments 
should go to the GitHub issue.

Mathias asked if the script would close the Bugzilla issues after 
migrating them, and Robert recalled we had disagreed on that in a 
previous discussion. Here, we agreed the issues should be closed 
after migration. Given the GH API rate limit, the migration will 
likely end up taking place over a few days. At the end of the 
migration, we can rerun the script if any new issues came in 
during that time and migrate those.

I didn't bring this up at the meeting, but we'll still need to 
get in touch with Brad to put our Bugzilla into read-only mode so 
that issues aren't reported there post-migration.

### Martin
Martin reported that he had started the beta of the next LDC 
release. At the time, he was hunting down some LDC-specific 
regressions (compiler crashes) that had shown up with the 
Symmetry code base.

He had discussed with Átila expanding the BuildKite projects 
being tested. Overall, the Symmetry code base has at least 100 
third-party dependencies on code.dlang.org. It's normal to hit at 
least some problems in those packages with a compiler bump. Some 
of the 2.101 problems had yet to be resolved. These might have 
been prevented if the problematic projects had been added to 
BuildKite.

An annoying bug he had run into before and recently encountered 
again is that Phobos on Windows doesn't properly support long 
paths. Windows has a special syntax for denoting long file paths 
that go beyond its old `MAX_PATH` limit of 260 characters 
(strings prefixed with `\\?\` are treated as long paths). This 
prefix is not required by the MS linker, by LLD, or by DMD. They 
handle it internally. Phobos doesn't. Where this is showing up is 
with dub. It uses a hash for paths to compiler output, and the 
hash takes up 64 characters of the 260-character limit. It's a 
real PITA. In his case, the LDC beta releases contain a `-beta-1` 
in the name, and with dub's hashes, the length ends up just over 
the limit. So to test things on Windows he has to fake the 
version just to make it work.

Martin filed [an issue for 
this](https://issues.dlang.org/show_bug.cgi?id=22059) last year, 
so if anyone wants to tackle it, you'll be plugging a Phobos hole 
that shouldn't exist in 2023.

Ali suggested that in the meantime, Martin might see if hard 
links can help him out if Windows supports them. He had 
discovered them recently and they solved a problem he'd had with 
file paths that contained problematic characters.

### Iain
Iain reported on a GDC regression someone had filed [as a dub 
issue](https://github.com/dlang/dub/issues/2577): GDC was failing 
to compile dub when warnings are configured as errors. He needed 
to track that down. The issue is still open.

He also noted that DMD 2.102 had been released two days before 
the meeting.

### Ali
Ali had nothing to report.

### Átila
Átila had nothing to report.

### Walter
Walter was having an annoying issue with `build.d`. It's required 
to build the compiler and build and run the test suite. 
Unfortunately, it requires a working Phobos to function. As it's 
currently set up, testing means compiling `build.d` with the 
compiler under test. This has caused him no end of frustration. 
`build.d` should be built with the D compiler and *not* the 
compiler under test. And other programs (like checkWhiteSpace) 
are being run before the test suite is executed which also 
require a working compiler. What should happen is that you build 
the compiler and then run the test suite before all those 
programs are run. When he builds a faulty compiler, 
checkwhitespace has crashed. It should never crash.

Dennis said that the test suite had grown into a mess over the 
years, making it difficult to change. We have several different 
CI services, each with its own scripts and configuration files, 
sometimes in a different repository. They have subtle differences 
in exactly what they do. The build script itself is over 2000 
lines. He finds that baffling. It's just feeding source files to 
the compiler to generate an executable. Why is it so complex to 
describe? He hopes someone can take the time to rework the build 
system. (We previously discussed this in our August meeting at 
DConf, where Iain brought it up.)

Walter asked again why the build compiler isn't used to compile 
`build.d` when testing. Razvan said it depends on which CI is 
running. Some use the build compiler, some use the compiler under 
test. Dennis said it's so hard to keep track of all of it. Walter 
said he runs the test locally and the problem crops up there. I 
noted that the build systems and tests were cobbled together by 
multiple people over time, and getting them straightened out is 
on our TODO list. Walter said he sometimes is ready to delete 
`build.d` and go back to using makefiles.

This took us into further discussion on problems with the build 
system and test suite, the proper way to build everything (e.g., 
test the compiler before building Phobos), how LDC handles it, 
etc. Everyone agreed we need to fix this. Mathias suggested the 
gist of the problem is that DMDlinks with `-lphobos` instead of 
`-ldruntime`. It should probably separate them as LDC does. 
Martin explained why he thinks using a D script to build the 
compiler is problematic in terms of cross-compilation and 
bootstrapping, and suggested we should get rid of `build.d`. We 
were running out of time, so we deferred this discussion to a 
later date.

Aside from that, Walter had been working on the top issues in 
Bugzilla that are causing people problems. One of the issues he 
felt he needed to fix was Rikki Cattermole's problems with 
`ModuleInfo`. Part of that means removing things from 
`ModuleInfo` that require `ModuleInfo` to be generated. One of 
those was a list of local classes that is built so that 
`Object.factory` could instantiate classes.

Martin agrees that this is work that should be done. The local 
classes thing just to support `Object.factory` is very ugly and 
prevents lots of otherwise unused symbols from being stripped 
from the final executable. He's also not very happy about the 
`ModuleInfo` dependencies on each other, which drags in lots of 
stuff. Especially `object.d` drags in so many modules with their 
own `ModuleInfo` dependencies, and those modules drag in more 
modules with their `ModuleInfo` dependencies... Whenever you use 
a tiny bit of DRuntime you're dragging everything in. He's not 
happy about that at all, but he doesn't believe it's a pressing 
issue.

Walter said that everyone has their own top issues with D and 
they're all different. He has to decide which ones to fix. (This 
is a good opportunity to remind everyone that I'm collecting the 
gripes and wishes at social at dlang.org precisely so we can figure 
out what emerges as the top issues. I have to say that so far, 
there is a very, very wide range of "top" issues. I can't know 
for sure until I sit down and collate all the input, but I 
haven't seen any specific problems mentioned by more than a 
couple of different people. Everyone really does have their own 
top issues.)

Martin said it was good that Walter had recently tackled a 
template emission problem. [He had submitted a 
PR](https://github.com/dlang/dmd/pull/14855) with a one-line 
change that made on template emission bug go away. Walter's 
initial comment was, "Can it really be this simple?" Martin said 
it wasn't *that* simple, as there were some cycle issues in 
trying to determine whether a template instantiation should be 
emitted. Martin says template emission is an interesting topic 
that needs to be tackled by people who really know the language, 
as opposed to the `ModuleInfo` stuff which only affects a tiny, 
tiny fraction of people.

This got Iain and Martin into a discussion about whether that 
one-line change is the right way to go, in which they went off 
into the weeds of the template emission implementation, 
particularly the problem with cyclic references. Walter said that 
we weren't going to be able to solve that problem in this meeting 
because we were short on time. That one-line change might not be 
ideal or the perfectly correct situation. But there are two 
things in favor of it: it passes the test suite and it's a small 
test case. By merging the PR, we get the test case into the test 
suite and a future solution is going to have to account for that 
test case. He'd had discussions with Átila where they agreed that 
one of the problems with solving template emissions issues is 
that no one had been able to produce small test cases. Here we 
have one, and we need to get as many small cases for template 
emission cyclic references into the test suite as we can, even if 
the PRs that bring them in aren't ideal solutions.

Martin's main concern is performance. Iain had done some 
refactoring that will likely have mitigated any performance loss 
the PR might have caused, but we don't have any way to measure it 
in the test suite. What he would like to do but had no time for 
was to test Walter's change on Symmetry's code base. They have a 
very interesting project that consists only of templates. It's 
always a challenge for each compiler bump. If Walter's PR passes 
that project as well, Martin's confidence in it would be much 
higher. Iain said it should also be tested against Weka's code 
base, so someone should ping Johan Englen in the PR thread. If it 
passes Symmetry and Weka, then Iain has no quarrels. Walter said 
that because it's a one-line change, it's very easy to back out 
of if things go wrong.

I interjected here to note that the `ModuleInfo` issue might not 
be a high priority problem, but it's a problem that I've seen 
people complain about in Discord. We have some tough problems 
that will require teams of people who understand the compiler and 
the language, as Martin said, but anything Walter can fix on his 
own will make *someone* happy. Besides, we don't have teams of 
people to solve the tough problems right now. We need to organize 
them somehow. I suggested that one step we can take to get there 
is to start bringing more people into our monthly meetings (not 
the quarterlies; we have too many people there already) in the 
same way we brought Mathias and Robert in. (They were previously 
just D users working in their own interests, but now they also 
work on tasks that arise in our meetings, and they enhance the 
range of perspectives on the issues we discuss.) So we should 
consider other regular contributors who we can bring in.

## The next meeting
As noted above, the next meeting is scheduled for March 3rd at 
14:30 UTC. As always, if you have any specific concerns, ideas, 
or prospects to discuss with us, please reach out and let me 
know. I'm happy to bring you into one of our monthly meetings to 
hear you out.