D Language Foundation January 2024 Quarterly Meeting Summary
Mike Parker
aldacron at gmail.com
Wed Apr 3 16:25:49 UTC 2024
The D Language Foundation's quarterly meeting for January, 2024
took place on Friday the 5th at 15:00 UTC. It lasted for about 45
minutes.
Our quarterly meetings are where representatives from businesses
big and small can come to bring us their most pressing D issues,
status reports on their use of D, and so on.
## The Attendees
The following people attended the meeting:
* Walter Bright (DLF)
* Iain Buclaw (GDC)
* Luís Ferreira (Weka)
* Max Haughton (Symmetry)
* Dennis Korpel (DLF)
* Mario Kröplin (Funkwerk)
* Mathias Lang (DLF/Symmetry)
* Vijay Nayar (Funnel-Labs.io)
* Átila Neves (DLF/Symmetry)
* Luna Nielsen (Inochi2D)
* Razvan Nitu (DLF)
* Mike Parker (DLF)
* Guillaume Piolat (Auburn Sounds)
* Carsten Rasmussen (Decard)
* Robert Schadek (DLF/Symmetry)
* Bastiaan Veelo (SARC)
## The Summary
### Luís
__Undefined references__
Luís said that Weka had encountered a lot of undefined references
when attribute inference infers the wrong attributes. For
example, sometimes `@nogc` was inferred when really the GC was
being used, or vice versa. Then when linking with other modules
where the attributes were inferred differently, they ended up
with undefined references.
They had a workaround---creating an alias to another template
with the mangled name that is supposed to be generated---but it
was very cumbersome. Right now, this was their biggest issue.
He said this didn't manifest when compiling with one compiler
invocation, but when multiple invocations were involved. They
would like to be able shift to building with LTO and compiling
the objects file by file rather than in one compiler invocation.
He also noted that one of the issues they had with compiling as a
single invocation was the amount of RAM used.
He said they also saw errors with attribute inference that were
dependent on how the compiler was invoked. Each of the files
should compile independently, but depending on the order they're
passed to the compiler or which ones are passed, the errors they
get are different. He said that basically, in a specific semantic
pass, the compiler was just giving up on attribute inference.
He also tangentially brought up a problem with [import order
causing forward reference
errors](https://issues.dlang.org/show_bug.cgi?id=23414).
Walter said he needed to see Bugzilla issues demonstrating the
problems before he could attempt to fix it. Luís said there were
some similar issues reported. He was trying to Dustmite down to
some minimal examples. Átila noted that it was very difficult to
isolate template issues like this. He'd run into similar issues
over and over and hadn't yet filed anything on Bugzilla because
it's was never a trivial example.
Walter said he understood and was aware of Átila's longstanding
problems with this. Robert said he'd never encountered these
problems at all, but that could be because of the way he used
templates. Atila thoought it came down to whether you compile all
at once or not.
Max said that Symmetry had gotten a reduced case at one point. He
thought it had made it into Bugzilla, saying Martin would have
filed it. That particular case may even have been fixed.
Walter reiterated that he needed a test case to reproduce it.
Talking about it wouldn't move anything forward. Luís said he had
a test case in the PR he had submitted in an attempt to fix it.
He noted that Walter had also made an effort, but that the PR got
stuck and then sank because it didn't solve the problem.
(__NOTE__: [Bugzilla Issue
#17541](https://issues.dlang.org/show_bug.cgi?id=17541) describes
the problem. The failed attempts Walter and Luís made to fix it
are in [dmd PR #10959](https://github.com/dlang/dmd/pull/10959)
and [dmd PR #15534](https://github.com/dlang/dmd/pull/15534).)
__Compile times__
Weka's biggest issues with compile times were with the
performance of template instantiations and CTFE. They use the
time trace flag with LDC and it shows these are the biggest
problems.
Luís asked if there were any plans to implement caching of
template instantiations so that they wouldn't need to be
instantiated on subsequent runs of the compiler. He also asked if
it would be possible for us to have a JIT to speed up CTFE.
Razvan said he was unaware of any plans to implement template
caching. Luís said the idea would be to do something like LDC
does with its codegen cache, where it caches the LLVM IR. Razvan
asked if he was suggesting caching the actual template instance
or the result after the template has been interpreted.
Luís suggested that the AST should be pure. Some work could be
done to separate out the member functions that mutate AST nodes
so that they could be made immutable. With such a pure AST, then
we could make it hashable and cache some of the work after the
semantics, after some code gen.
Razvan said there had been some discussion of this in the past,
but he was unaware of anyone working on it.
__Linter update__
Luís said that after many attempts of implementing a linter in
the compiler, he had been succssful running LDC-lint over the
Weka code base. They only had two rules enabled for the moment.
He planned to enable broader rules eventually, but there were too
many false positives now. But down the road, he hoped this would
be a great thing to show off, having a linter built in to the
compiler as opposed to being an external tool.
__Huge stack allocation__
Luís said that Weka's code used a lot of fibers. They'd found
that `opAssign` on huge structs [generated huge stack
allocations](https://issues.dlang.org/show_bug.cgi?id=23868#c2)
when it shouldn't. On their LDC, they had an attribute to see big
stack allocations at compile time so they could detect those, but
they ended up either disabling `opAssign` or overriding it to not
use stack allocations.
Mathias asked if it was because the parameter wasn't passed by
reference. Luís said he wasn't sure. This was the `opAssign`
generated by the compiler. Maybe it wasn't being passed by
reference and should be.
(__UDPATE__: I asked Luís to email me later with the Bugzilla
links related to the problems he raised along with any other
relevant details. He did, and included a couple of other things
he'd forgotten to mention in the meeting, one of which was
another problem that [prompted Walter to open a Bugzilla
issue](https://issues.dlang.org/show_bug.cgi?id=24343). The
issues with undefined references and compile times are still
unresolved, but Walter closed the stack allocation issue as
WONTFIX, saying the existing algorithm was the pedantically
correct one. Generation of `opAssign` could be turned off with
`@disable`, and changing the algorithm could be done by
overriding `opAssign`.)
### Carsten
Carsten said everything was working okay. They had been
optimizing some stuff and had some problems, so they had moved
away from libp2p (they were using the Go implementation) to
another library written in C. They wanted to have everything in
clear code.
Now all their code was in D and C, and that was nice. At the
moment, they were precompiling all the C code and then linking it
in, and they'd had some issues with memory leaks. What they
wanted to do was to use the C code via ImportC, so getting that
working was their next step.
Other than that, they were profling to find where their time was
going.
Walter asked him to keep us posted on their experience with
ImportC. Carsten said they only had three C dependencies, and
having everything in D and C was much nicer to work with.
Carsten then reported that they'd had their server running for a
month and were working on improving it. They'd had one crash that
came from someone sending in malformed data that they hadn't been
prepared to handle. They were currently working on being able to
seamlessly swap out nodes. He said they had a lot of work to do,
but at least now they were in a position to have fun doing it.
Basiaan asked which compiler they were using. Carsten said they
were using DMD for development because of its faster compilation
and LDC for deployment. Bastiaan said he'd had trouble using
ImportC with LDC, so he was using only DMD for now. It was
probably a preprocessor issue. Carsten said they hadn't fully
incorporated ImportC yet and were only experimenting with it. He
had considered doing the preprocessing first. It was a learning
process.
### Bastiaan
Bastiaan reminded us of the big problem regarding multi-threaded
performance that he'd brought up in [the October quarterly
meeting](https://forum.dlang.org/thread/jucppgelbdvxvoubxzkr@forum.dlang.org). He noted that he'd replied to my summary of that meeting with [a report that they'd since solved the problem](https://forum.dlang.org/post/rftxnpxalyeiruolkoto@forum.dlang.org). He said they were very pleased with that.
He said it had been frustrating because there were multiple
causes. Every time they'd thought they'd found *the* cause, it
didn't have any effect at all until they'd fixed all the causes.
The three major problems are outlined in his forum post. Once
they solved those, they saw good improvement.
### Mario
Mario said there was nothing new from Funkwerk. Mathis Beer was
still on holiday, but would probably have something for the next
quaterly meeting.
### Guillaume
Guillaume said he didn't have many issues to talk about because
everything worked.
One problem he'd had was with newer releases of LDC. He does
consumer software, but with LDC he couldn't target old versions
of macOS. It wasn't a big deal as he could just mix and match the
binaries.
He also wanted to reiterate that better support for Mac silicon
and WebAssembly would eventually be a good idea, but there was
nothing really hardcore urgent he could think of.
He also wanted to thank everyone. He had filed 64 bug reports
over the course of his involvement with D. 45 of them had been
solved and the others weren't that relevant. He thought that was
excellent bug coverage that he could enjoy.
### Max
Max had to leave for a work meeting shortly after I called on him
and said he would try to make it back if we were still going
later. He didn't, as the meeting didn't last much longer beyond
this point.
### Vijay
Vijay said he was very happy with D and how it functioned, but he
had a couple of questions that had come up during a recent small
project.
__Question about message passing__
First up, he'd noticed that when you were doing message passing
between threads using the `send` and `receive` functions, you
almost invariably had to duplicate the data because you could
only pass immutable objects. But when you were no longer using
the object in that thread, he wondered if this would result in a
copy or if it was being optimized away.
Átila said the options were to either pass immutable data, which
he knew could sometimes be a pain, or `shared` data, which was a
bigger pain. But the only other way he knew to get around that
was some form of ownership in which you could pass ownership from
this thread to the other. Even though the data would be mutable,
you wouldn't be able to use it in this thread anymore, so it
would be safe. He said the Pony language had something like that.
But that would mean some form of isolation in the D language.
__Question about Ddoc__
Second, when documenting template parameters vs. regular
paramters, was there a separate segment for template parameters?
In the language and the code, they were distinct, but in Ddoc
there was only `Params:`. Mathias said that putting them all in
`Params:` handled it.
Vijay asked if grouping them together was the right choice given
that they were distinct in the language and the code. Mathias
thought so. He said that at Sociomantic (where they had
maintained their own fork of the compiler), they'd implemented
`Template_Params:` at one point. He said from his experience with
that, there wasn't much value in it. If we really wanted to
separate them, the compiler could figure it out and we could just
display them separately. But he didn't see the value in it.
Átila agreed. He said that you knew the names, the names weren't
going to repeat, so you could probably tell them apart.
### Luna
Luna said her current blocker was the lack of ObjectiveC support
in LDC. She was considering making a pull request for it.
Another issue was that DRuntime on macOS was [using a function
that Apple didn't
allow](https://github.com/ldc-developers/ldc/issues/4547) to be
present in executables released on the App store. She was also
planning to submit a PR to LDC to fix that. Carsten said they'd
had the exact same problem on iOS. He didn't know which function
it was or what the fix was, as his colleague had handled it.
Walter asked what function it was. Luna said it was something in
the dylib API. Apple didn't allow anything using private APIs in
the App store. She was trying to figure out how to make that
work. It was related to how the GC scanned executables and shared
libraries for spots to search. You had to use a different API for
that.
Walter said he hadn't been aware of that at all, and he'd
appreciate a write up on what was wrong or what the correct API
was. Luna said she'd filed an issue on the LDC repo. She said
this was an LDC-only issue. She hadn't seen that the API was
called on the DMD version of DRuntime. Walter said that was
something for Martin to handle, then.
(__UPDATE__: Luna subsequently [fixed the dylib
issue](https://github.com/ldc-developers/ldc/pull/4558).)
### DLF Team
Robert, Mathias, and Iain all had nothing to report for this
meeting, though Mathias said he was happy with the language.
Walter said he'd been working on improving ImportC and addressing
bug reports. He was always interested in hearing if people were
finding success with ImportC and if it was fulfilling its mission
to save them time. He had nothing else for this meeting.
Luís asked if Lua was compilable with ImportC. Walter said he had
no idea. Luís said he had a use case for it so would give it a
try. I speculated it probably would be because it's a clean
codebase with wide support across compilers and platforms. Luís
said it didn't have a lot of macros.
Carsten said in their experiements with ImportC, they'd run into
the problem where module names clashed when two C files had the
same name. But they were able to work around it by using soft
links that combined the directory name and file name. Walter
asked if he could do a write up about their approach and then we
could add it to the documentation for ImportC. He said this was
what he called an impedence mismatch between D and C, so he'd
been collecting the workarounds people used because other people
were going to run into the same things. Carsten said he could
show Walter his script.
Dennis said he'd been working on named arguments and had made a
forum post about some issues he'd encountered that could use some
feedback. He then said he'd had more success with ImportC. Most
of his bugs had been fixed for TreeSitter to compile. There was
just one more designator issue that was open, but apart from that
he could now use ImportC to compile it.
## The Next Meetings
Our January Monthly meeting took place the following Friday. We
also had a planning session on January 19th. The next quarterly
meeting is happening on April 5th.
More information about the Digitalmars-d-announce
mailing list