D vs Go in real life, part 2. Also, Erlang.

[Atila Neves]

It all started here:

http://forum.dlang.org/thread/lmkvmfpysiokpqgsynar@forum.dlang.org

Not wanting to be outdone, my buddy Jeff (the Go guy) went and 
wrote a new benchmark. My implementation didn't do as well on 
that one so I went back to the code and optimised it. Also, our 
other buddy Patrick from work decided to get in on this and use 
it as an excuse to learn Erlang. So now 4 different 
implementations in 4 different languages are duking it out.

There are two benchmarks, both written in Go. The first is 
loadtest and it measures throughput. I varied the number of 
client connections to the server and measured how many messages 
were sent per second.

The second benchmark is pingtest and it measures latency. The 
quantity I varied here was the number of wildcard subscribers 
(it'd take a while to explain what they are to anyone who isn't 
familiar with MQTT). It has a bunch of client connections talk to 
each other, and instead of trying to send as many messages as 
computerly possible they wait for the acknowledgement from the 
partner connection. The results are as follows (I tried making 
charts but LibreOffice wasn't helping and then I got bored), with 
the numbers being thousands of messages per second:

loadtest
Connections:   100            500            750            1k
D + vibe.d:    121.7 +/- 1.5  166.9 +/- 1.5  171.1 +/- 3.3  167.9 
+/- 1.3
C (mosquitto): 106.1 +/- 0.8  122.4 +/- 0.4   95.2 +/- 1.3   74.7 
+/- 0.4
Erlang:        104.1 +/- 2.2  124.2 +/- 5.9  117.6 +/- 4.6  117.7 
+/- 3.2
Go:             90.9 +/- 11   100.1 +/- 0.1   99.3 +/- 0.2   98.8 
+/- 0.3

pingtest
wsubs:         20             200            400
D + vibe.d:    50.9 +/- 0.3   38.3 +/- 0.2   20.1 +/- 0.1
C (mosquitto): 65.4 +/- 4.4   45.2 +/- 0.2   20.0 +/- 0.0
Erlang:        49.1 +/- 0.8   30.9 +/- 0.3   15.6 +/- 0.1
Go:            45.2 +/- 0.2   27.5 +/- 0.1   16.0 +/- 0.1

So, D was faster than the other contenders by far in throughput, 
2nd place losing to the C implementation on latency. I'm still 
not sure why that is. Profiling in this case is tricky. I'm 
pretty sure the profiler is still ticking away when a fiber 
yields - the top function is the one that reads from the network, 
which I can't do much about.

GC was pretty much a non-issue, which I'm both surprised at and 
happy about. I allocated as much memory as I wanted at first, and 
by the time I tried to optmise memory usage by allocating less 
all I managed to eke out in performance terms was a paltry extra 
8% or so.

All in all though, so much for his "nothing is going to be faster 
than Go because this is what it's good at". If he'd measured it 
against mosquitto first and realised it was already losing to C I 
probably wouldn't have written an MQTT broker in D, so I guess 
that was a good thing. :)

Atila

P.S. vibe.d is awesome, although I wish it'd compile with ldc or 
gdc