Structured logging (was Early review of std.logger)

Johannes Pfau nospam at example.com
Tue Oct 15 08:20:56 PDT 2013


I think one increasingly important point for std.log is 'structured
logging'.

Structured logging is basically not simply logging textual messages, but
also logging additional KEY/VALUE pairs of data. The idea is that logs
should not only be readable by humans but also easy to parse and
analyze. Structured logging often also includes UUIDs to simplify
finding similar errors/problems in a log file.

For example: Instead of logging
logf("Couldn't login as %s on server %s : Reason: %s", user, server,
      errorCode);

we'd do:
log(LOGIN_FAILED_UUID, "Couldn't log in", ["User": user, "Server":
    server, "ErrorCode": errorCode]); 

The log can then be queried for all events with 'LOGIN_FAILED_UUID'
uuid, Server="..." ErrorCode=42, etc.

As a nice benefit structured logging can also log the function, module
and line of the source file that logged the message, Exceptions can be
written to the log in a nice way, etc.

SystemDs journal [1] is a structured replacement for syslog and is
already being used on some linux distributions (ArchLinux, Fedora).
It's likely that the journal will replace syslog on linux so at least
for server-side software structured logging support is becoming
important. The GNOME guys are also working on a log viewer for
systemd's journal [2]. 

std.log certainly doesn't need to provide a backend for the journal
right now. But some questions regarding structured logging need to
taken into account:

* Will structured logging be integrated with the normal logging at all?
* Will the public API somehow collide with the proposed std.log API?
* Should the API for log backends already support structured logging or
  is it possible to add this later on?

Most important:
* What to do about structured logging calls when logging to a simple
  text logger? Ignore the additional fields or serialize them? What
  about the UUID?

Even if we don't have a backend implementation it'd be nice to have the
public API available. Otherwise new projects cannot use structured
logging with std.log right now and the user code needs to be
rewritten once std.log does support structured logging.

[1]
http://0pointer.de/blog/projects/journalctl.html

[2]
https://mail.gnome.org/archives/gnome-announce-list/2013-September/msg00097.html


More information about the Digitalmars-d mailing list