Asynchronous Programming and Eventhandling in D

[chmike via Digitalmars-d-learn]

On Wednesday, 6 July 2016 at 11:33:53 UTC, Eugene Wissner wrote:
>
> The only reason libev was choosen is that it is the simplest 
> implementation I know about. A few C files. I had an 
> educational purpose: I wanted to see how an event loop works on 
> low level. Asyncio was for me no-go, since I've not written a 
> lot in C++, and can only read the code a bit. So I'm not 
> hanging on libev. The only another implementation would be the 
> python event library. It is also pure C code and it was much 
> more cleaner written than libev on the first sight.

Your project and work is valuable on many aspects. I didn't mean 
to depreciate it.

> Now there are two problems with my work:
> 1) The first is something we all are tired to talk about: 
> manual memory management. I make a proof of concept and am 
> writing the code that is 100% marked as @nogc. It has side 
> effects. For example I allocate exceptions and thay should be 
> freed after catching - it is something, phobos doesn't do. As 
> said it is an experiment; I would like to see how it works.
>
> 2) Performance. The main reason I started the writing at all is 
> that the existing implementations seem to have only performance 
> as criterium. Performance is super important, but not on the 
> cost of design, usability and extensibility. For example in 
> vibe.d (and libasync) everything possible is defined as 
> structs, everything that would be interesting to extend is 
> final; and after it you go to phobos and see a "workaround". 
> For example Mallocator is a struct but you have to be sure, it 
> is an allocator. How do you force the right interface?

That is a valid point. I know it is hard to get the best 
performance and optimal API design at the same time. The reason 
the methods are final is to avoid the overhead of virtual method 
indirection.

> static if (hasMember!(Allocator, "deallocate"))
> {
>    return impl.deallocate(b);
> }
> else
> {
>    return false;
> }
>
> Somethinkg like this would be ok for C. But for a language with 
> interfaces, it is ugly design independent of how it performs. 
> Everything I say here is IMHO.

This would mean we need a new design pattern that supports both.

> Except these two points I'm interested in some kind of 
> collective work aswell. It is very difficult as one man job. I 
> didn't know other people are also working on similar 
> implementations. Nice to know.
>
> Are you aware of any benchmark tools in other languages that 
> could be used?

The benchmark tools available are mainly testing web servers. And 
the exact operation pattern is not very clear. One of such 
benchmark tool is wrk [https://github.com/wg/wrk] that measure 
HTTP request speed. The problem is that it measure the 
performance of the I/O and HTTP handling.

Here are some benchmark results:

* https://www.techempower.com/benchmarks/
* https://github.com/nanoant/WebFrameworkBenchmark

How can D be worse than Java ? 

My strategy would be to split the problem. First get the async 
I/O optimal. Then get HTTP handling optimal, and finally get 
database interaction optimal (the optimal async I/O should help). 
An investigation on the methods used by Java/undertow to get 
these performances could help.


I would suggest to implement a benchmark client doing some 
predefined patterns of I/O operations similar to web interactions 
or the most common types of interactions. And a server 
implemented in C using native system I/O operations. This server 
implementation would then be our reference.

What would be measured is how much slower our different D 
implementations are relative to the C reference implementation. 
This will allow us to have a benchmarking test that doesn't 
depend that much of the hardware.