[phobos] std.event / event loop for phobos

[Johannes Pfau]

 On 08.09.2010 16:02, Masahiro Nakagawa wrote:
>
> I am thinking about std.event.
> This module supports system-dependent APIs(kqueue, epoll, etc...) and
> abstraction layer.
>
I think the best solution would be a wrapper / abstraction layer for
libev. All of the mentioned APIs (kqueue, epoll, select...) are broken
in some way, so writing a new event loop system in D will be painful and
it will take a long time until it gets stable. Also libev already has
support for more event sources (timer, periodic, signal, stat, ...).

Quoting libev documentation:

EVBACKEND_SELECT
Not /completely/ standard, as libev tries to roll its own fd_set with no
limits on the number of fds, but if that fails, expect a fairly low
limit on the number of fds when using this backend.
This backend maps |EV_READ| to the |readfds| set and |EV_WRITE| to
the |writefds| set (and to work around Microsoft Windows bugs, also onto
the |exceptfds| set on that platform).

EVBACKEND_POLL
It's more complicated than select, but handles sparse fds better and has
no artificial limit on the number of fds you can use (except it will
slow down considerably with a lot of inactive fds).

EVBACKEND_EPOLL

The epoll mechanism deserves honorable mention as the most misdesigned
of the more advanced event mechanisms: mere annoyances include silently
dropping file descriptors, requiring a system call per change per file
descriptor (and unnecessary guessing of parameters), problems with dup
and so on. The biggest issue is fork races, however - if a program forks
then /both/ parent and child process have to recreate the epoll set,
which can take considerable time (one syscall per file descriptor) and
is of course hard to detect.

Epoll is also notoriously buggy - embedding epoll fds /should/ work, but
of course /doesn't/, and epoll just loves to report events for
totally /different/ file descriptors (even already closed ones, so one
cannot even remove them from the set) than registered in the set
(especially on SMP systems). Libev tries to counter these spurious
notifications by employing an additional generation counter and
comparing that against the events to filter out spurious ones,
recreating the set when required.

EVBACKEND_KQUEUE
Kqueue deserves special mention, as at the time of this writing, it was
broken on all BSDs except NetBSD (usually it doesn't work reliably with
anything but sockets and pipes, except on Darwin, where of course it's
completely useless). Unlike epoll, however, whose brokenness is by
design, these kqueue bugs can (and eventually will) be fixed without API
changes to existing programs. For this reason it's not being
"auto-detected" unless you explicitly specify it in the flags (i.e.
using |EVBACKEND_KQUEUE|) or libev was compiled on a known-to-be-good
(-enough) system like NetBSD.

While stopping, setting and starting an I/O watcher does never cause an
extra system call as with |EVBACKEND_EPOLL|, it still adds up to two
event changes per incident. Support for |fork ()| is very bad (but sane,
unlike epoll) and it drops fds silently in similarly hard-to-detect cases

EVBACKEND_PORT

This uses the Solaris 10 event port mechanism. As with everything on
Solaris, it's really slow, but it still scales very well (O(active_fds)).

Please note that Solaris event ports can deliver a lot of spurious
notifications, so you need to use non-blocking I/O or other means to
avoid blocking when no data (or space) is available.

While this backend scales well, it requires one system call per active
file descriptor per loop iteration. For small and medium numbers of file
descriptors a "slow" |EVBACKEND_SELECT| or |EVBACKEND_POLL| backend
might perform better


-- 
Johannes Pfau

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/phobos/attachments/20100908/b2a36397/attachment-0001.html>