Design Philosophy

The core idea of Logger is inspired by the 12 Factors App design principles of treating logs as an event stream, and letting the execution environment handle log routing. Although the library does allow the flexibility of providing custom outputs, the author highly recommends the "Separation of Concerns" benefits that come from simply outputting structured logs to stderr. This approach was found to be highly effective in a high-traffic production environment; console logs were simply routed from Docker to rsyslog to Kafka, then sent to multiple destinations, including third-party observability platforms, local time series databases for metrics dashboards, as well as being filtered and sent to a data warehouse for business analytics.

(Unlike the 12fa manifesto, we prefer logging to stderr over stdout, as the system is already helpfully providing two differentiated streams. It seems a shame to pollute a potentially useful "output" stream with logging!)

The Tracer design is an attempt to extend the 12 Factors principles to observability, partially as a visceral response to OpenTelemetry's complexity. As mentioned, the author had benefited from decoupling service implementations from log routing, so introducing a whole new layer of "environmental dependency" at the app tier was not desired. It is also an attempt to drastically reduce the "cognitive load" required for leveraging tracing - rather than adding a whole new complicated system, the idea was "what's the smallest extension to logging that will be compatible with distributed tracing".

The approach envisioned is that the logs that contain trace data will be routed to a collector somewhere in the log processing pipeline (example code that parses logs and forwards to OpenTelemetry is provided).

There are legitimate concerns as to whether traditional logging infrastructure is well set up for the potentially much higher demands of tracing. As such, it is hard to recommend Tracer for high-traffic production environments until more tooling and best practices are established.