What makes a language popular - and how to manufacture D popularity

Sat May 28 08:41:57 UTC 2022

I have followed parts of the thread "Why is D unpopular, redux" 
here on these forums,
and decided to contribute my 5 cents on the matter.

I am a 20+ year Java developer and have seen Java's rise in 
popularity, then fall, and recently a resurgency in popularity 
again. The same with JavaFX. I have seen languages
such as JavaScript and Python creep into spaces where Java used 
to be the language of
choice, and wondered why that happened.

I first encountered D about 5-7 years ago, and started reading 
the book "The D Progamming Language". I was sold almost 
immediately! At almost every feature I felt "Yes! That is exactly 
how I would like a language to work and look (syntax)".

So - why did I not end up becoming a D developer? And why has D 
not become more popular?

Here is what my interpretation of what attracts developers to a 
language - aside from the language features themselves:

Language Popularity Factors:
============================

Ease of Use:
------------
Lots of developers learn JavaScript and Python because they are 
really easy to get started
with. Just install Python or Node.js, and you can start making 
small programs. Also, they allow for easy scripting - no need for 
classes, main methods or even IDEs to get started.

Other languages are reasonably easy to get started with too.

Maybe D needs to emphasize what is needed to get started with D 
easily?
Maybe also what is required to get a full development environment 
up, incl. an IDE with syntax highlighting, a compiler etc. ?
I know that info is available on the D website, but it could be a 
bit more "visible". Maybe a "Get Started" section somewhere?

Use Case:
---------
Developers often choose a language because they need it
for some specific use case.

- I need to do data science, therefore I choose Python (lots of 
data science libs).
- I need to write a game, therefore I choose C++ (or C# with 
Unity)
- I need to make an iOS app, therefore I choose Swift.
- I need to make an Android app, therefore I choose Java / Kotlin.
- I need to make a web app, therefore I choose JavaScript (+ 
maybe Java / C# for the backend).
- Etc.

What is the use case where D shines? If there isn't one - should 
the D community maybe
pick one, and provide excellent tooling, libraries and products 
for that?

Ecosystem:
----------
It is not enough to have a cool language. You need a cool 
ecosystem of open source APIs
addressing a wide variety of problems. And - these APIs must be 
well documented.

The D ecosystem was the primary reason I did not start developing 
in D. I simply could
not find the APIs I needed at the time, for the project I was 
working on. So, I stuck
with Java - which was not perfect - but good enough.

Jobs:
---------
Developers often learn a specific language because they need it 
for their job, or because the hope to be able to get a job 
working with that language.

Jobs are of course reasonably tightly correlated with use cases 
and ecosystem.

Should the D website have a jobs section?

Popularity:
-----------
Humans are social animals to some degree. If we do not feel 
confident making a choice ourselves, e.g. because we lack the 
knowledge to make a proper judgment, we tend
to look at what others have chosen.

Unfortunately we cannot magically make D more popular overnight.
There are other ways to gain popularity though, some of which I 
list below.

Social Gravity and Manufactured Momentum:
=========================================
No human being has complete knowledge of every product - e.g. 
every programming language
out there. Therefore, no human being can make a fully calculated 
decision about what product to choose.

Instead, humans tend to gravitate towards products (and 
organisations, movements etc.)
they perceive as having momentum. This is also known as "social 
validation".

Perceived Momentum Signals:
---------------------------
Momentum is typically perceived via the following primary signals:

A) How often does a person hear / read about X ?
B) How many people does the person know talking about X?
C) How often does X have new releases, new features etc?

All of these signals can be artificially manufactured. I give 
some examples of how a bit later.

Perceived Momentum vs. Other Language Popularity Factors:
---------------------------------------------------------
Perceived momentum is a strong gravitational force. Perceived 
momentum tend to translate into both validation and anticipation. 
You feel validated in your choice and anticipate all the exciting 
develoments in the future - because X has a strong perceived 
momentum.

Developers can forgive a language for many issues, if the 
perceived momentum of that language is high. The perceived 
momentum makes them anticipate that it will certainly
be better in the future.

Some complain that D has bugs - but all languages have some level 
of bugs. Developers
can forgive that if the most serious bugs are fixed fast, and 
others soon.

Some complain that D is not always backwards compatible, but many 
languages are not 100% backwards compatible. C# has had several 
non-backwards compatible releases. Python broke backwards 
compatibility between the 2.X and 3.X version series. Even Java 
which has focused a lot on backwards compatibility is now slowly 
removing features and APIs, and making subtle syntax changes.

C#, Java and Python are still pretty popular despite sometimes 
breaking backwards compatibility - because developers anticipate 
that the result, and the future, will be better all in all.

Some complain that D doesn't have all the latest, cool features 
of language X, Y and Z.
But neither does any other language. C++ does not have all C#'s 
features. Java does not
have all of D's features. C# does not have all of Python's 
features. Neither does
Python, JavaScript, Rust, Ruby, R etc.

Once a language has "enough" features to make a developer 
reasonably productive,
other factors start to play a bigger role, such as use cases, 
ecosystem, jobs -
and perceived momentum.

Here are some examples of how Java increased its popularity again
(that D could perhaps learn from):

Java's Resurgency:
------------------
Java was for several years considered "dead" - even if lots of 
enterprise software was
still written in Java, and there were lots of Java jobs. Why was 
Java perceived as "dead" then?

The primary reason was, that Java evolved too slowly. Languages 
such as Scala and later Kotlin had more advanced features and 
evolved quicker. Even C# started to distance Java.

To fix that, Java started adding some of the more popular 
features from Scala. But that still wasn't quite enough. Java was 
still evolving too slowly and unpredictably. You did not know 
when the next Java release was out, nor what features it would 
end up containing.

To fix the perceived slow evolution, from Java 10 and forward, 
Java changed to a 6 month
fixed release cadence. New major version of Java drops every 6 
months, quite precisely.
Whatever features are ready at that time, make it into the 
release.

The 6 month fixed release cadence made Java feel like it is very 
much alive.
You *know* new features will drop soon. Maybe not your favourite 
feature - but some feature will be added. This addresses momentum 
signal C) in the list above.

The new release cadence also has the side effect of creating 
anticipation and excitement.
What will be included in the next release? Luckily, you won't 
have to wait too long to
find out - and to be able to play with it.

The new release cadence of Java also has a related side effect. 
Every 6 months there is
a lot of buzz around Java - because of the new release. Articles 
are written and shared
on social media about the new features in the latest / coming 
Java release.
This addresses momentum signal A).

And - to be fair: The new features added to Java these days are 
quite advanced - often
addressing performance, multi-core development, polyglot 
development etc. These are features that are useful where Java is 
often used - in server side development, and
big data engineering. It's not all just manufactured buzz.

Lately Java has started to focus on compiling to native 
executables. This is useful
for targeting smaller devices, and to reduce cold start on AWS 
Lambdas (Function as a Service).

JavaFX Growing Popularity:
--------------------------
JavaFX is the newest of the Java GUI toolkits available. First, 
it lived outside
of Java. Then from Java 8 it was included as part of the Java 
SDK. Then from Java 11
it was removed again. The removal made Java developers think that 
JavaFX was dead.
However, JavaFX was primarily removed to make the core Java SDK 
smaller.

JavaFX is still very much alive today, and its popularity seems 
to be growing,
based on number of SDK downloads at least. Here is why (in my 
perception):

JavaFX adopted the same 6 month release cadence as Java. This has 
brought JavaFX
the same signal advantages as Java. JavaFX feels alive, buzz is 
generated regularly, and people start anticipating new releases 
with excitement.

Additionally, the JavaFX community has honed in on a specific use 
case: Cross platform GUI app development. More specifically:

- Cross platform desktop apps.
- Cross platform mobile apps.
- Browser execution of JavaFX.

That means, that JavaFX can run on Windows, Linux, MacOS, iOS, 
Android and Raspberry Pi (Linux) - and in the browser. This makes 
JavaFX one of the most cross-platform GUI toolkits available out 
there - apart from game engines such as Unity.

GraalVM:
---------
GraalVM is a virtual machine from Oracle that is capable of 
running Java, JavaScript, Python, Ruby, R, LLVM and WebAssembly. 
Being able to run Java, GraalVM can also run Scala, Kotlin, 
Groovy and Clojure.

GraalVM can run these languages either separately, or inside the 
same application.
This way, GraalVM is trying to solve the language fragmentation 
problem of polyglot organisations. Use Python, JavaScript, Java 
etc. when that is convenient - call into LLVM when you need 
performance or need to call a C library.

GraalVM can also compile your polyglot app to native executables 
and many other things.

GraalVM has added greatly to the Java ecosystem buzz - and make 
developers feel less
"captured" in the Java ecosystem.

What can D learn from this?
===========================
I did not write this post to promote Java. I wrote it to explore 
how D could learn
something from Java's resurgence in popularity.

A) If D does not have a fixed and regular release cadence, D 
might be able to benefit
from adopting that.

B) If D has one or more core use cases where it is really strong 
- emphasize them.

If not, perhaps the D community could choose one or more to 
emphasize?

Preferrably use cases that organisations with deep pockets care 
about. Financial
institutions don't care about Python. They care about data 
science - and Python
addresses that need.

C) Manufacture buzz. Write blog posts. Submit talks to 
conferences.
Solve interesting problems and talk about it.

D) D won't make everyone happy. There will always be some that 
complain about
something, or prefer some other language.

A Possible Trend Toward Low Level Coding - For D to "Latch on To":
----------------------------------
For a while, the trend in development (as I perceive it) has been 
towards increased
developer convenience. Use easier, interpreted languages such as 
JavaScript, Python or Ruby.
Use game engines such as Unreal Engine or Unity instead of coding 
your own.

However, the last few years some developers have been raising 
some alarm bells
about this development. Fewer and fewer developers know how 
computers and languages
work at the lower levels. The result is slow, bloated software. 
But more importantly:

Who will write the next generation of Node.js, the Python 
interpreter, the Java VM,
the game engines, device drivers etc. , if fewer and fewer people 
know how to
develop low level code?

Not many developers yet talk about this, but the concern has been 
raised by Casey Muratori
in his video "The Thirty-Million Line Problem" :

https://www.youtube.com/watch?v=kZRE7HIO3vk

Also other people and videos address this problem, although it is 
still just a few people making noise about it.

Could D jump on to that trend, and start marketing itself (among 
other things)
as the "high level low level" language of choice for the future?

Why I am looking at D - again:
==============================
There are two primary reasons why I am looking at D again, 5-7 
years after leaving it:

1) Enticed by the concern about fewer people knowing low level 
coding, I have started
looking into it again. This time mostly out of general interest - 
not because I need
D to solve a specific problem. The choice was between C, C++, 
Rust and D (so far).

2) I am working on a hobby / art / research project attempting to 
create a "smart media platform".
This project involves:

- A smart media player (desktop / mobile app).
- A smart media fabric node (infrastructure to connect media 
player apps).
- A P2P topology designed for media distribution.

- A custom binary data format
- A custom binary network protocol for RPC and Subscribe-Notify
- A custom VM + script language (for simple scripting of the 
platform)
- A custom stream storage engine
- A custom stream processing API
- Possibly some RSync implementation

- Compression algorithms.
- Encryption algorithms.

- Possibly a "remote desktop" type protocol too.
- 2D graphics - and possibly 3D graphics in the future.

I will develop the prototype in Java, because I know I can get 
most of it done with Java + JavaFX combined. Over time I would 
like to replace parts of it with a more low level (higher 
performing) language. Here I am hoping D could be that language.

I am the kind of developer that appreciates convenience, but 
prefers control - if I have to choose between control and 
convenience. In Java I get at decent tradeoff between the two, 
but would like the control I could get in D.