How I use D

[Cym13]

The original "Phobos is cool" thread encouraged me to share my 
experience.

D has been one of my two main languages for about 5 years now. I 
use it almost daily in my work where it has proved very useful. 
Yet I don't think a case such as mine is discussed often so I 
thought I would take the time to explain how and why I use D. I 
don't expect anything in return, but I wonder how many users have 
a similar use of the language and whether this user base is 
significant enough to warrant consideration.

But first of all, why should you care?

You probably shouldn't. I'm not a software developer, my 
company's business doesn't rely on D in any way, I'll never write 
a killer app or case study and even if I were to share a library 
it probably wouldn't be maintained more than a week. I'm not 
someone that'll bring much value back in the language.

I'm a security consultant. My job is to find security issues and 
exploit them. I'm sure most of you know about pentesters yet my 
job goes a bit beyond common penetration testing. I review 
websites and networks but also attack electronic devices, perform 
malware analysis, source code review, cryptographic review, human 
manipulation, lockpicking... Anything goes.

In that context I very often need to write dedicated tools. I 
don't have the luxury of time: since most missions take about a 
week I feel lucky if I get 2 days to write an exploit. This means 
that I need to prototype a solution very quickly. For that, the 
goto language in my field is Python (Go is coming big recently). 
Python is my other language of predilection and fast scripting is 
where it absolutely shines and given its large library support 
(both standard and community based) it is often the right tool 
for the job. However I often end up needing more performances or 
low level access and optimizing Python is hard.

For example when trying to exploit a tight race condition I need 
all the speed I can get, especially when that race condition 
happens in a chip on which I'm directly connected. Similarily for 
things like bruteforcing running 10% faster may be the difference 
between finding an access in time or not.

The difference with D is that I can start very scripty, using no 
explicit type, then adding details in once I have a strong 
skeleton. I know that style is often dismissed by more industrial 
users but it's one of the core reasons why I use D.

I use almost exclusively UFCS function chains when prototyping 
because they fit the way I think perfectly. UFCS is absolutely a 
killer feature for me from a design perspective. If I use 
something else it is either because I'm optimizing (I found that 
basic for loops often yield better performances than long 
templated UFCS chains) or because I'm porting another program to 
D.

Since I don't choose the technologies I'm facing I end up 
constantly butting heads against new formats or protocols. In 
these cases I often port part of a parsing library to D or use a 
C one directly if available.

D is perfect to use C libraries or port libraries from other 
languages.

I rarely get enough time to port a full library but the fact that 
D supports all paradigms really shines there since it enables me 
to port programs without rethinking their architecture. It's 
rarely optimal but it's a huge time saver as a first step.

I also get to use that at a low level. For example the other day 
I was studying a ransomware. I needed to do some statistics over 
a large amount of files, started decompiling the malware code by 
hand translating it into C-style D, then hit a function that was 
not fun to reverse (often the case in cryptographic code). I took 
the disassembly, stuck it into an asm{} statement, tweaked it 
just enough to get it working and continued decompiling the rest 
without needing to worry about that function anymore.

I could have done it in another way, but this meant that I didn't 
need to change language at any time, it was all in D, I could 
optimize it all as needed at the end. This is especially 
important since I need to balance this low-level code with the 
high-level contexts in which they are found (and said context 
rarely makes it easy for me).

There are some tools that I use often and wrote myself in D 
(generally variants of "we need to try every element of this list 
as quickly as possible"). In general it is painless and 
efficient. I've even had a few surprises of code that I 
prototyped in D, reformated in C-style D for performance, 
translated in C for even more performance, discovered that pure C 
was slower than my C-style DMD generated code (?) so rolled back 
to D.

The only time where I had some issues were that time I wrote a 
Windows trojan in D. I needed to do some pretty convoluted stuff 
to load my code in just the right way to avoid antiviruses and it 
wasn't fun to do it in D.  Cross-compiling this proved another 
hurdle. It would have been easier in C++ directly. But I can't 
very well blame anyone not to make malware development easier and 
none of the issues I had should impact normal development.

Most of the code I write exploits a specific cryptographic 
mistake, buffer overflow or race condition for a specific client 
so I have nothing left to publish. As I said at the beginning, 
even though I write D code daily I am just not working on 
projects that are likely to ever bring back to the community. 
And, for all the code I write, I'm not a developper. I wouldn't 
know how to manage a project that lasts more than 3 days 
honnestly. That doesn't mean that I'm not serious about code 
quality (I spend days reviewing code and explaining to 
programmers how to make it better after all) but I don't 
contribute code to open-source projects. I'm just a selfish user 
minding his own business.

As for issues... (What? It wouldn't be a correct forum post 
without issue...)

The low number of native libraries is not ideal. Yes, I can use a 
C library or port it from another language but I've generally got 
less than two days to work on a project. Any time spent on this 
is time not spent on actually solving the problem.

For the same reason I am a huge advocate of battery-included 
stdlib. I know many are of the opinion of moving as much as 
possible from Phobos into dub.  For me any library that isn't in 
dub is one more library I have to evaluate, try, keep track of 
when they update. It's lost time. Huge projects can have a 
mindset of "we'll manage our own version anyway" but my projects 
are written in a few days and are useful a week or two at most. 
So go Phobos, go! We also need lots of libraries but that is the 
kind of things that is built upon a strong standard library, not 
at its expense.

Then there's the omnipresent negativity of a huge chunk of the 
community...  it's rough. I used to read the forums daily until I 
noticed that it stressed me a lot, made me feel angry and sad and 
that it wasn't the place I wanted to be. Now I mostly keep an eye 
on Announce. I have no idea how this can be fixed though so I'll 
leave it at that. I completely assume the irony of posting this 
here.

All in all I've tried hard to find a better language than D for 
my work, but its design choices make it a really good fit for my 
purpose. The main reason why I still use lots of Python is 
because of its library support and because I can expect my 
coworkers and clients to be able to run it without issue or 
hour-long explainations. I hope it stays that way even though I 
understand why most trends I see on the forum target more 
"industrial" users (by which I mean working on month or year-long 
software development projects in a professionnal setting). I have 
no expectations from that post, but I wonder how many others use 
D everyday at work in a similar fashion.