7  logging: Write logs

7.1 What’s logging and why use it?

Many of us insert print() statements to track the progress of our programs or debug an issue.

# data_processing.py

# print statements inserted in various parts of the code
print("Loading data...")
print("Data loading failed.")
print("Data loaded successfully.")

print("Starting the data processing...")
print("Missing data found in column 'id'.")
print("Unexpected value encountered in column 'id'.")
print("Data processing completed.")

While simple to use, print() statements have several limitations:

  • It doesn’t record a message’s level of importance (e.g., info, warning, error).
  • It doesn’t provide a timestamp for when an event occurs.
  • It lacks flexibility in directing output to different destinations (e.g., console, file).
  • Too many print statements can clutter the output, making it hard to read.

This is where logging comes in. Logging is the process of recording messages that describe events occurring during the execution of a program. A typical logging system captures these messages and provides mechanisms for filtering, formatting, and directing them to various outputs. These messages can provide valuable information for debugging, monitoring, and auditing purposes.

In Python, the built-in logging module provides a flexible framework for emitting log messages from Python programs.

7.2 Using the logging module

To get started with logging in Python, you need to import the logging module and configure it according to your needs. Here’s a basic example:

# data_processing.py
import logging

# configure the logging system
logging.basicConfig(
    format="{asctime} | {levelname:<8} | {message}",
    style="{",
    datefmt="%Y-%m-%d %H:%M",
    level=logging.INFO
)

# log messages at different severity levels in various parts of the code
logging.info("Loading data...")
logging.error("Data loading failed.")
logging.info("Data loaded successfully.")

logging.info("Starting the data processing...")
logging.warning("Missing data found in column 'id'.")
logging.critical("Unexpected value encountered in column 'id'.")
logging.info("Data processing completed.")
2025-11-17 16:07 | INFO     | Loading data...
2025-11-17 16:07 | ERROR    | Data loading failed.
2025-11-17 16:07 | INFO     | Data loaded successfully.
2025-11-17 16:07 | INFO     | Starting the data processing...
2025-11-17 16:07 | WARNING  | Missing data found in column 'id'.
2025-11-17 16:07 | CRITICAL | Unexpected value encountered in column 'id'.
2025-11-17 16:07 | INFO     | Data processing completed.

In this example, we configure the logging system to

  • Use a specific format for log messages that includes the timestamp, log level, and message.
  • Set the date format for timestamps.
  • Set the logging level to INFO, which means it displays all messages at this level and above.

The predefined logging levels in increasing order of severity are DEBUG, INFO, WARNING, ERROR, and CRITICAL. (See Logging Levels from the official document for more details.)

If you want to direct log messages to a file instead of the console, you can specify the filename parameter in the basicConfig function:

logging.basicConfig(
    filename="data_process.log",
    format="{asctime} | {levelname:<8} | {message}",
    style="{",
    datefmt="%Y-%m-%d %H:%M",
    level=logging.INFO
)

7.3 Learning Resources

For more information on using the logging module and best practices for logging in Python, refer to the following resources:

  • Logging in Python: A comprehensive tutorial on logging in Python, covering basic to advanced topics.
  • Logging HOWTO: The official logging HOWTO guide.

A popular alternative logging library is Loguru. Compared with the built-in logging module, Loguru offers a simpler and more user-friendly API, making it easier to set up and use.

For more information on Loguru, refer to the Loguru documentation. The following tutorials also provide a good introduction to Loguru: