NGINX Unit and vibe.d Integration Performance

[Kyle Ingraham]

Hi there,
I'm looking for help with the performance of an integration I'm 
trying to write between NGINX Unit and D. Here are two minimal 
demos I've put together:
- https://github.com/kyleingraham/unit-d-hello-world (NGINX 
Unit/D)
- https://github.com/kyleingraham/unit-vibed-hello-world (NGINX 
Unit/vibe.d)

The first integration achieves ~43k requests per second on my 
computer. That matches what I've been able to achieve with a 
minimal vibe.d project and is I believe the max my benchmark 
configuration on macOS can hit.

The second though only achieves ~20k requests per second. In that 
demo I try to make vibe.d's concurrency system available during 
request handling. NGINX Unit's event loop is run in its own 
thread. When requests arrive, Unit sends them to the main thread 
for handling on vibe.d's event loop. I've tried a few methods to 
increase performance but none have been successful:
- Batching messages when sending new request messages to minimize 
overhead. This increased latency and didn't improve on throughput.
- Using vibe.d channels to pass requests. This achieved the same 
performance as message passing. I wasn't able to use the channel 
config that prioritized minimizing overhead as the API didn't 
jive with my use case.
- Using a lock-free queue 
(https://github.com/MartinNowak/lock-free) between threads with a 
loop in the vibe.d thread that constantly polled for requests. 
This method achieves ~43k requests per second but results in 
atrocious CPU usage.

~20k requests per second seems to be the best I can hit with all 
that I've tried. I know vibe.d can do better so I'm thinking 
there's something I'm missing. In profiling I can see that the 
vibe.d thread spends a third of its time in what seems to be 
event loop management code. Am I seeing the effects of Unit's and 
vibe.d's loops being 'out-of-sync' i.e. there being some slack 
time between a message being sent and then being acted upon? Is 
there a better way to integrate NGINX Unit with vibe.d?