How do you use D?

[Laeeth Isharc via Digitalmars-d]

On Friday, 28 July 2017 at 14:58:01 UTC, Ali wrote:
> While the Orgs using D page is very nice ... I hoping to hear 
> more personal stories ...
>
> So
>
> How do you use D?
> In work, (key projects or smaller side projects)
> in your side project, (github, links please)
> just to learn something new? (I would easily argue that 
> learning D will make you a better C++ programmer, maybe not the 
> most efficient way, but I a sure it i very effective)
>
> Did you introduce D to your work place? How? What challenges 
> did you face?
>
> What is you D setup at work, which compiler, which IDE?
>
> And any other fun facts you may want to share :)

I started programming in 1983: BBC BASIC, 6502 assembler, Z80 
assembler.  I learnt to program C on an Amstrad PCW running CP/M, 
and compiler I used had K&R style declarations.

Then I discovered economics, and my life took a different course. 
  I moved into trading and managing money, but I always programmed 
on the side to help me solve investment and business problems.  
Kept it quiet because programming was for a long time low status 
and clashed with what people expected to see in a money manager.

Late 2013 I recognised that the way our business used technology 
was completely broken.  The only way to make the most of 
technology is to combine an understanding of investing, financial 
instruments, the investment business, and technology in one mind. 
  But where am I going to find a guy like that?  So I looked in 
the mirror and realised I had to brush up my skills.

So I started building tools to help me invest.  My friends mostly 
thought it was a crazy course of action, because it's rare if you 
leave investing even temporarily to return to it, and I love 
markets, but sometimes the direct approach isn't the right one.  
We're drowning in data but don't have good tools to make sense of 
it.

I learnt python and cython, but kept looking, because I wanted to 
have my cake and eat it.  Why can I not have all of productivity, 
performance, static typing/correctness, abstraction, and code 
readability - I don't think I should have to choose just a 
couple, and I am not going to. That led me to D in 2014.

At school they used to ask us if everyone else jumped out of the 
window  would you do it too? And it's a profitable approach in 
financial markets to develop and learn to trust your own 
judgement of things.  If a highly respected and very 
knowledgeable economist tells you "you do realise that there is 
no basis in economic theory for what you are suggesting", you 
need to be able to revisit your thinking, see what you might be 
missing, but in the end trust your own judgement over that of the 
putative expert.  And he subsequently wrote a very impressive 
explanation after the fact of how what "couldn't be justified in 
theory" did in fact happen.  And it's a bit similar with 
programming language choices and such.  Its way better to appeal 
to people who make up their own mind and bear the consequences 
then to those who have to cover their behinds by getting the 
right ticks in the boxes because the are never going to be 
earlier adopters except through some unfortunate accident - 
because you also don't want such people as early adopters!

Since then, one thing led to another, and I ended up hiring a few 
people from the community to help me as consultant developers.  
Maybe a bit more than 10% of Team Phobos, based on a simple 
calculation.

I have also ended up running technology, amongst other things, 
for a decent size hedge fund.  I am using D for the project I 
started beforehand, and we are starting to explore its usefulness 
in the rest of the firm for some core analytics.  It pays to 
start small and build on early successes.

Has D been good for me? What have been the consequences of the 
French Revolution? In both cases it's too early to say with utter 
confidence since life is complicated and things are not always 
what they seem to be in the moment, but I have a view on the 
latter, and I think the answer to the former is definitely yes.

Finance used to be relatively at the leading edge.  The industry 
got a bit too fat, it hired the wrong people as it expanded too 
quickly, and then after the crisis people had other things to 
worry about - first survival, and then an exploding burden of 
compliance.  At one big bank even the compliance people complain 
about the number of compliance people they have.  On top of that, 
there's so much legacy systems that it can be very difficult to 
do anything creative or new.

So as an industry we fell behind a bit, and when you go through a 
cycle like that if becomes somewhat self-reinforcing.  To turn 
things around you need to hire very good people, but it's not so 
easy to find very good people who are willing to work with the 
people who created and tolerated the mess in the first place.  
But it can be done, and I think based on a view from afar that 
the banks will do better on this front than they have been.

For a firm like the one I am involved with its different because 
we are at a post startup stage.  Some legacy but not that much, 
and it's possible to do new and creative things.  And it's also 
possible to use languages that might not win on social factors if 
based on intrinsic factors it's by far the best answer for us.  
And we don't need to make a big bet, because we can make a start 
in a small way for us, and build on successes.

One of the biggest benefits from D is I think a cultural one.  
We're in an age of specialists and of the full stack overflow 
engineer, with people strapping together ready made components.  
If that's all programming is about, you don't need to have people 
with flair, because it really flattens out the differences 
between people.

But in a business that's highly dynamic, there's also a cost to 
such a conventional approach.  One loses some ability to adapt to 
unanticipated change, and you lose coherence in understanding 
about the total behaviour of your system, and about what the 
choices are and what the implications might be.  This 
fragmentation of knowledge that accompanies micro-specialisation 
isn't entirely healthy.  And maybe if working with framework X 
for years will make you faster then someone who hasn't experience 
of framework X, speed of development isn't from a commercial 
perspective the only, or sometimes the main factor, in 
determining productivity, which is not a materialist measure 
(features accomplished per 40 hours), but a measure of economic 
value added per unit input, and value added relates in this case 
to long lived capital, and part of the value of capital relates 
to optionality and the ability to adapt what you have built to an 
unexpected need, or to change the way you do things in a way you 
didn't originally plan for but that's better.

In other words you can think about the plasticity of D from the 
point of view of real options theory.

When I started my career there was an understanding that whilst 
experience had value, someone capable could often pick things up 
given a few pointers and a chance to learn.  We somehow forgot 
that in the years since then, though this way of thinking is 
making a comeback.

So a benefit of D is that because it attracts people who aren't 
put off by the tooling etc, one tends to find people who are 
adaptable and don't mind getting their hands dirty and doing 
something they never did before because that's what is the most 
valuable right now from a business perspective.  Lack of polish 
can itself also be a highly effective filter.

Many seemingly hairy things are not that bad with a bit of 
experience.  But if you don't get such experience because you 
spend your life in an environment where things just work then 
when something doesn't work it's a bit more painful.

Suppose there's part of an application that's slow.  I simply 
can't imagine finding a good and experienced D developer who 
wouldn't have put it through a profiler and doesn't already have 
some idea anyway of bits of the code that are slow.  In some 
other communities, it really isn't like that at all!  And 
similarly if you think the GC is a problem in your code somewhere 
it's just normal to use the instrumentation to check and to use 
an allocator if that's the case.  Not something you would even 
give a lightning talk about doing in D land, because it's just 
what you do.  For some managed languages the norm is quite 
different.

London used to be a set of villages.  But today any former 
village that has an underground station has lost its character, 
whereas those that don't are harder to get to and have somehow 
preserved it more.  And it's the same with virtual communities - 
the very difficulty initially of gaining entry can also preserve 
its character.

Benefits of D? The easiest reasons to adopt something are not 
necessarily the reasons that ultimately matter most.  Some 
proximate attractive aspects of D are the ability to write native 
code in a modern language that can be called from many different 
languages with wrappers being generated via attributes.  Excel-D 
is one such tool, but there are other languages we need to target 
also.

There's a rising interest in alternative data sources in finance. 
  Numerical data is not that big.  Text data can be, if not big, 
uncomfortably large vs what we are used to working with.  And D 
is fast with strings, but that's not the case for some other more 
modern languages used in a finance environment.

Code clarity is worth something too.  It's easy to read 
well-written D code, to some extent even if you don't know D.  
Some other languages not so much, because of the fragmentation of 
OO code and the verbosity and use of abstraction that isn't doing 
any useful work but does constitute another layer in the way of 
understanding what's going on.

I personally use Sublime or vim on Linux.  Others use Visual D.

Biggest challenges? People don't have reference points for D.  It 
doesn't slot into an existing mental category. "so you're saying 
D is good for larger data sets".  Well, that's true  but it's not 
quite only what I am saying.

Other challenges - normal human anxiety about doing something 
different. And habit - apart from necessity, the most powerful 
force in the universe.  If someone is in pain, they are going to 
listen to something that might be an answer. C++ programmers I 
work with aren't sceptical of the GC, but they're excited about 
doing things better because they both love C++ and appreciate 
that it has flaws.

Windows experience is not that great.  Part of it is just native 
code on Windows rather than D, but expectations are set by 
languages that do everything for you.  Most of the time 
installing a library in python is one command.  In D you might 
have the dub wrapper or binding and then the C library itself.  
And installing the latter is not always just one command.

Would be great to have a web front end to D irc, because some 
companies have to be careful about what chat services are 
accessed from their systems. Of course I can for me set this up 
myself but others might not bother to do that.

Thinking about my own experience and that of another commercial 
user I wonder if there is a market for paid support for compiler 
plus phobos plus certain libraries.  The first adopter within a 
firm might be quite ready to persevere, but the others they 
follow may do better if there is someone they can call when they 
get stuck.  Not necessary for my group, but in time some might 
find it valuable.  Not very difficult but just need patience and 
ability to communicate.

Maybe some value also to training eg python users to show them 
how they can do similar things in D  and what the benefits are. 
People learn in different ways and I think it's a minority that 
learn from reading books unless very motivated.