btdu - a sampling disk usage profiler for btrfs (written in D)

[Vladimir Panteleev]

On Tuesday, 10 November 2020 at 09:40:33 UTC, Jacob Carlborg 
wrote:
> On Sunday, 8 November 2020 at 17:23:32 UTC, Vladimir Panteleev 
> wrote:
>
>> - D programs that build fine on one Linux machine may still 
>> fail to build with mysterious linking errors on another, even 
>> when using Dub which takes care of dependency management. I 
>> saw two counts of this, caused by differences in DMD/LDC and 
>> Arch/Debian (one being that, for whatever reason, libz is not 
>> pulled in on LDC/Debian despite being a Phobos dependency). 
>> Also, LDC is the D compiler that's installed by default when 
>> the system wants a D compiler (e.g. if you try do install Dub 
>> by itself).
>
> I don't think this is specific to D. I've seen in the past 
> problems caused by package maintainers not building the package 
> in the same way as upstream. Or they split up a package in 
> multiple packages.

I think it might be less of a problem in e.g. Go.

>> - The garbage collector is still a major hindrance for system 
>> programming. In this case it was due to the ioctls used being 
>> slow, and when the GC tries to stop the world to do its thing, 
>> it just hangs the entire program until ALL ioctls in all 
>> threads complete.
>
> You should probably never let the GC run on a realtime thread, 
> like audio or video processing (not sure if ioctls falls into 
> this category). These days, modern UIs should probably fall 
> into the realtime category.

Doing UI without GC in D would be pretty painful.

But, by itself the GC doesn't add much latency to introduce 
stutter in the UI - a GC scan is generally quick enough that the 
UI doesn't feel laggy or stuttery. The problem is that the GC is 
waiting for all threads to finish their ioctls, while the program 
otherwise is completely suspended. This affects not just UI, but 
throughput.

>> - One user wondered why the program needed so many threads. 
>> The answer was that half of them were owned by the GC (it 
>> never stops its worker threads, they just sit idle).
>
> Is that the answer? I mean, the GC doesn't create any threads 
> by itself, does it?

Yes, it does, since the introduction of parallel heap scanning in 
2.087:

https://dlang.org/changelog/2.087.0.html#gc_parallel

>> - I used the Deimos ncurses bindings package. I'm thankful 
>> that it already existed, though I had to push some fixes to 
>> fix static linking. The most annoying part was waiting 
>> overnight for code.dlang.org to pick up the new tags, because 
>> there is no way to get it to update a package unless you're 
>> the owner, and no way to otherwise specify a dependency unless 
>> using a branch (which is deprecated and prints a big warning 
>> when your users build your program).
>
> Since 2.094.0, you can specify a Git repository as a dependency 
> [1]. You can also specify a local path as a dependency [2], 
> useful when developing a library and an application at the same 
> time, as two separate Dub packages.
>
> [1] https://dlang.org/changelog/2.094.0.html#git-paths
> [2] https://dub.pm/package-format-sdl.html#version-specs

This is super useful. Thanks.