What is your favorite D feature?
H. S. Teoh via Digitalmars-d
digitalmars-d at puremagic.com
Fri Jun 23 17:33:48 PDT 2017
On Thu, Jun 22, 2017 at 12:48:25AM +0000, Seb via Digitalmars-d wrote:
> Hi,
>
> I am currently trying to modernize the D code example roulette on the
> dlang.org front page [1]. Hence, I would love to hear about your
> favorite feature(s) in D.
> Ideas:
> - favorite language construct
> - favorite code sample
> - "only possible in D"
[...]
Today, I just got another idea for this: show off std.parallelism,
especially parallel foreach!!!
Here's the background: I have a program that takes user input, converts
it into D code using a code template, invokes dmd to compile it into a
shared library, and then loads the shared library so that the main
program can call the compiled code directly. The output is a series of
images that can be assembled into an animation. The first version of
the code looked like this (greatly simplified, of course):
void main(string[] args) {
auto input = parseUserInput(args);
auto libhandle = compile(input);
foreach (i; 0 .. n) {
auto outfile = File(...);
... // do stuff
libhandle(...); // call compiled code
... // more stuff
outfile.rawWrite(image);
}
}
When n is large, of course, this runs rather slowly, because it's
generating the images one by one. Parallel foreach comes to the rescue:
void main(string[] args) {
auto input = parseUserInput(args);
auto libhandle = compile(input);
foreach (i; parallel(iota(0, n))) { // ***
auto outfile = File(...);
... // do stuff
libhandle(...); // call compiled code
... // more stuff
outfile.rawWrite(image);
}
}
The only change is marked in // *** above. It's just a 1-line change,
and boom, now the loop automatically spawns worker threads and generates
the images in parallel. Instant 6x speedup on my AMD Hexacore machine!
This all sounds very obvious, but actually there are some D-specific
features at play here that make this even remotely possible:
- First of all, I deliberately described the shared library bit, because
at first I wasn't sure if std.parallel would Just Work(tm), the reason
being that the shared object has global variables, and the loop body
does change the values of those global variables. After reading the
warning about implicit data sharing between threads in the docs, I was
a bit wary of writing the loop as a parallel loop to begin with.
Thankfully, TLS comes to the rescue: because the shared object is
compiled as D code, the globals sit in the TLS segment, rather than
C/C++'s default global storage. So modifying those globals turned out
to be completely harmless w.r.t. parallel foreach: each thread would
have its own copy of the global, so they don't interfere with each
other. More importantly, this holds not only for the main program,
but also for the *shared library* loaded via dlopen(). (It was this
last part that I wasn't 100% sure about; it was already obvious that
if these globals were in the main program, it would be no problem by
default. But for this to also extend to manually-loaded shared
libraries was a big relief for me.)
That was the most tricky bit, but then other D features come together
to make it possible to change a linear loop into a parallel loop by
basically just editing the foreach statement:
- opApply: in spite of being the scorned illegitimate second cousin of
the beloved range-based counterparts, this is what made it possible to
keep the only difference between a "native" foreach and a parallel
foreach just a matter of a simple syntactic change in the foreach
aggregate (just add a call to `parallel`). Without this, we'd have to
hoist the loop body out into a delegate or some such, and replace the
foreach with an awkward call to some obscure function in std.parallel.
Not horrible, but just lots of fussy little editing details to take
care of. Thanks to opApply, all we needed to do was to essentially add
`parallel` to the one line. The compiler takes care of converting the
loop body into the requisite delegate, transparently close over the
required local variables, etc.. No fuss, no muss.
- iota: the underappreciated construct, without which we'd have to
manually construct some indexing array or some other such thing for
parallel() to work with. I.e., a lot more tedious typing than just a
simple 1-line change.
- Finally, parallel() itself: one of the most ingenious API designs I've
ever seen, it nicely encapsulates the dirty details of managing
TaskPools manually, spawning worker threads, assigning Tasks to
threads, synchronizing with threads afterwards, and all of that fussy
boilerplate that one has to drown in if doing this in a language like
C or C++.
Thanks to all of the above, D lets you switch from a linear loop to a
multithreaded loop just by changing 1 line of code. I don't know of any
other imperative language that can boast this. (Of course, functional
languages, esp. the pure ones, have this "by default". But we're talking
about a dirty, mutating, imperative language with loop bodies that
modify *global* variables -- to do this in spite of said mutation of
globals, now *that's* an achievement.)
T
--
English is useful because it is a mess. Since English is a mess, it maps well onto the problem space, which is also a mess, which we call reality. Similarly, Perl was designed to be a mess, though in the nicest of all possible ways. -- Larry Wall
More information about the Digitalmars-d
mailing list