JSON logging in Python - Andrey Zhukov's Tech Blog

A lot of the log systems (loggly, logentries, sematext, kibana, etc.) can understand the JSON format. The benefits: it’s easier to search, filter and analize logs.

Here is an example of configuration for Python with structlog library.

Install

pip install structlog
pip install python-json-logger

Update logs config

import logging.config

from structlog import configure, processors, stdlib, threadlocal

logging.config.dictConfig({
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'json': {
            'format': '%(message)s %(lineno)d %(pathname)s',
            'class': 'pythonjsonlogger.jsonlogger.JsonFormatter'
        }
    },
    'handlers': {
        'json': {
            'class': 'logging.StreamHandler',
            'formatter': 'json'
        }
    },
    'loggers': {
        '': {
            'handlers': ['json'],
            'level': logging.INFO
        }
    }
})

configure(
    context_class=threadlocal.wrap_dict(dict),
    logger_factory=stdlib.LoggerFactory(),
    wrapper_class=stdlib.BoundLogger,
    processors=[
        stdlib.filter_by_level,
        stdlib.add_logger_name,
        stdlib.add_log_level,
        stdlib.PositionalArgumentsFormatter(),
        processors.TimeStamper(fmt="iso"),
        processors.StackInfoRenderer(),
        processors.format_exc_info,
        processors.UnicodeDecoder(),
        stdlib.render_to_log_kwargs]
)

The output will be like:

import structlog
log = structlog.getLogger(__name__)
log.info("event", some_param=42)

{"message": "event", "lineno": 1, "pathname": "<ipython-input-8-0783dd619691>", "some_param": 42, "logger": "__main__", "level": "info", "timestamp": "2017-07-27T12:59:22.440132Z"}