D and the FOSS ecosystem

[Joseph Rushton Wakeling]

On 04/04/12 12:54, Iain Buclaw wrote:
> There's actually more than one side of the argument here other than
> the one you raise, that are creeping up recurring topics alley.
>
> On the note of integration into the FOSS ecosystem, I'm firing off
> technical review of gdc for inclusion sometime later today.

Since it may be useful, I thought I'd make a handful of notes on the issues I've 
faced, as a fairly committed (but not fundamentalist) FOSS user, in getting on 
with D.

I can't remember exactly when or how I discovered D; I think it was probably 
some time in late 2007 or through 2008.  I was something of a novice in C++ at 
this point (I'd always worked in C) and was very attracted by the C++-like but 
clearly much more attractive and effective syntax and design.

The major concerns I had in using D covered a range of practical, technical and 
ethical issues.  Some of what I say here may come down to not having read up on 
the latest D progress -- my apologies in advance for any unfair critiques.


** Availability of D support for mainstream Linux distros **

This actually covered two different concerns: one was the packaging and ease of 
availability of DMD for different mainstream Linux distros; the second was FOSS 
compiler support and development roadmaps.

Packaging was a requirement not just for my own convenience but as an indicator 
of how easily I could get other people involved in contributing to any project I 
might start.  Even if I could handle a more complicated install, I didn't want 
anyone else to have to.

FOSS compiler support was for me a long-term requirement but not an immediate 
short-term one; I'd have happily used DMD as a stopgap so long as there was a 
clear sign of strong development activity on an up-to-date FOSS compiler to be 
arriving in the near future.  At the time I was first engaged with D, this 
really didn't seem to be the case; GDC seemed to have been abandoned and had 
only D1 support, LDC was being maintained but didn't seem to have any commitment 
to work on D2.

This concern has been substantially resolved by the new GDC team's excellent 
work -- I have to thank Vincenzo Ampolo / Goshawk for the blog updates that kept 
my interest in place.  The clear statements of intention and support from Walter 
Bright on getting GDC accepted into the core GNU Compiler Collection were also 
extremely important.

I made a deliberate choice to avoid attempting to use GDC until the new 
D2-supporting version was properly packaged and available, something which was 
partially resolved with Ubuntu 11.10 and is now fully resolved with Ubuntu 
12.04.  I don't know to what extent GDC is available in distros beyond Ubuntu 
and Debian, mainly because my Linux use is been pretty myopic in this respect ;-)

AFAICS the LDC version in Ubuntu 12.04 remains one with only D1 support; it 
would be good to see activity here, as a broader range of FOSS options both 
reflects the health of the D community and ensures there are multiple options 
available in the event that support for one project falters.

It's always been a concern that mainstream development support for D did not 
seem to be centred around a FOSS reference implementation.  Beyond the ethical 
aspects is a practical concern -- virtually all of the widely-adopted new 
languages of the last 10-20 years seem to have been built around FOSS.


** Performance of D code **

Most of my coding is for number-crunching simulations.  The last time I tried 
working with D, I found that DMD-compiled code was running at about half the 
speed of equivalent C++.  IIRC this was suggested to be because DMD's 
floating-point operations were not optimized to the same degree as g++.

A couple of casual tests in the last days with the same codebases suggest that 
GDC produces code of comparable speed to g++.  Which is nice. :-)

I'll be keeping an eye on this, obviously, but it looks like it may have been 
resolved for GDC at least.  I have a memory that the earlier gdc2 package in 
Ubuntu 11.10 built code that still ran a fair bit slower -- were any 
floating-point or other speed optimizations brought in recently?


** Easy use of C or C++ libraries and other languages **

As a relatively new language D was not as well-supplied with library packages 
compared to those I was used to using with C/C++ (or, where supplied, these were 
not incorporated into Linux distros).  At first glance at the documentation, it 
looked like this might not be an issue as it ought to be possible to use 
existing C and maybe C++ libraries.

I was keen on using D with the GNU Scientific Library (GSL), but found this 
impossible to do -- it turned out that the process of putting in place D 
bindings for the library was incredibly complicated and in-depth.  I'm sure this 
was partially inexperience and lack of understanding of what was necessary, but 
it was very offputting given that the documentation stressed that D worked well 
in combination with C.

It wasn't clear to me whether I needed to construct full bindings for the 
library or whether I could do this for the minimal set of library functions that 
I was calling.  In the event I never tried because (i) I wanted to build full 
GDC bindings anyway and (ii) the D/C++ speed differences mentioned above made it 
feel not worthwhile to keep pushing with D for the specific project I was 
engaged in, even if I could have limited my hooks to a handful of GSL functions.

Obviously the optimal situation would be for D to interface seamlessly with 
C/C++ libraries without the developer having to do anything on a per-library 
basis; I have no idea of the extent to which that's feasible.  What is still a 
concern is the lack of D libraries or bindings to popular libraries in C/C++ and 
other languages, available as packages in mainstream Linux distros (e.g. QtD, 
similar support for GTK+, Boost, ...).

The general case here is of course getting D to hook into different languages 
and I'm not sure of what the possibilities are.  IIRC Swig offers support for D, 
but as I didn't have a very nice time hooking together C++ and Python using 
Swig, I'd welcome simpler options.


** Summary **

D being a language with limited adoption, my main concerns in using it were the 
extent to which it would be easy to (i) get other people involved in a D-based 
project; (ii) avoid reinventing or rewriting the wheel in terms of 3rd-party 
libraries; and (iii) have performance comparable to that I could already get 
with C/C++.

What that means in practice is having at least 1 well-supported FOSS compiler, 
together with a broad collection of library functionality, either via dedicated 
D libraries or bindings to libraries in other languages.  All of this needs to 
be packaged for mainstream Linux distros.

This is partly a matter of me being lazy (I want to _use_ the language, not deal 
with setup problems) but also a matter of being confident it's a language that 
will be easy for others to adopt.  This is why e.g. even though I'm unlikely to 
write GUI applications, I'd like to see Qt/GTK+ support packaged, because I know 
that plenty of other people _will_ want that kind of support in place.

Anyway, all this has gone on for a fair length of time, so I think I'll stop 
here -- hope it was all useful info.

Thanks and best wishes,

     -- Joe