Logger Configuration to Log to File and Print to STDOUT

python

logging

handlers

configuration

byAnton Shumikhin·Aug 18, 2024

Configure Python's logging module for simultaneous logging to both file and stdout. Set up a logger, include a StreamHandler for console output, and a FileHandler. Here's the core of this setup:

import logging
import sys

# Setup logger and handlers. Like a band preparing for a gig!
logger = logging.getLogger('my_app')
logger.setLevel(logging.INFO)
console_handler = logging.StreamHandler(sys.stdout)  # stdout, not stderr! stderr is so... err.
file_handler = logging.FileHandler('app.log') # For those who love to keep records.

# Create a formatter. This is the stylist for our logs.
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')

# Attach the formatter to handlers. The band is styled!
console_handler.setFormatter(formatter)
file_handler.setFormatter(formatter)

# Add both handlers to the logger. Stage… set!
logger.addHandler(console_handler)
logger.addHandler(file_handler)

# Ready to rock and roll!
logger.info("This message is rocking both stdout and the file.")

In short, initialize a logger, tweak its level with setLevel(), establish handlers StreamHandler(sys.stdout) and FileHandler, apply a Formatter, and addHandler() to the logger. Now, your logs will simultaneously output to the console and a file.

Unpacking the details

Let’s take a deeper dive into the components and configurations:

The art of formatting

Formatter lets you dictate how your logs appear, including custom attributes for added context like timestamps and log levels.

Log rotation with style

Incorporate RotatingFileHandler or TimedRotatingFileHandler to ensure log files don't grow out of control (logorrhoea, anyone?). They rotate logs based on size or after certain intervals.

from logging.handlers import RotatingFileHandler

# A handler that knows when to take a break!
rotating_handler = RotatingFileHandler('app.log', maxBytes=10240, backupCount=5)
rotating_handler.setFormatter(formatter)
logger.addHandler(rotating_handler)

Separate stage for console and file shows

Set different levels for console and file logs – like setting stage volume for the audience (console) and recording (file).

console_handler.setLevel(logging.ERROR)
file_handler.setLevel(logging.DEBUG)

Adding color to your logs

Add a splash of color to differentiate log levels, using libraries like colorlog.

Reset before you go

Before setting basicConfig, clearing root handlers will prevent echoing the same logs.

logging.root.handlers = [] # Empty the room before the show!
logging.basicConfig(handlers=[file_handler, console_handler], level=logging.INFO)

Advanced configs and error handling

Dive deeper into the logging module for more nuanced configurations:

Choice of method

The logging.basicConfig function has a quick preset, but manually adding handlers lets you customize more for your logging jam.

Pitfall maneuvers

Duplicate logs: This might be due to multiple handlers sticking their straws in the same drink—your logger. Ensure only one handler is attached.
Color compatibility: Not every environment amicably accepts colorized output. Windows users might need the colorama package.