Logging in Python
Any application or script needs logging to share information with the user. The simplest logging method is the print function to write temporary logs. A permanent log file is a more general and valuable solution.
A log file is a text document.
An application can be seen as a room filled with people, where each person represents a module in the application. Without any logging, the room is in absolute silence which, more often than not, can drive a person mad.
Code generates logs to communicate with people.
All applications benefit from good logging. In a bad logging scheme, the room will be filled with people talking over each other. In such scenarios, it is better to disable all logging.
Logging methods are available in all major programming languages. Python provides a default logging package which is sufficient for nearly all tasks.
To make logging easy and flexible, the python logging package provides the following features,
Logger Objects
The logging package provides a logger object which can be used for separate log messages from different modules. A single log file can contain information from different application modules. Logger objects enable the people (modules) in the room (application) with the ability to speak (log messages).
Logging Levels
Not all log messages are equal — WARNING and ERROR messages are more important than INFO and DEBUG messages. Log levels create a hierarchy of importance for logging. People in the room can scream and shout if they require attention. The importance of the messages is defined based on the application.
Handler Objects
Handlers define the medium of communication. To write to a file, we use FileHandler, for stdout/stderr we use SteamHandler. These are the most commonly used handlers. People in the room can choose to write instead of speak.
Formatter Objects
The log messages can be set to follow a standard structure. For example, each log entry should always begin with a timestamp. The format of the log depends on the application’s need — there is no format that is universally better than other formats. Each person in the rooms decides to follow a common language structure — subject-verb-object. Anyone who reads the log file will find it convenient if every line follows a fixed structure.
In my experience, keeping two separate files at DEBUG and INFO levels is convenient in the long-term. The INFO log file provides a timeline of the major events during execution. The DEBUG log file contains messages useful for debugging and the INFO level messages. The following function creates such a logger which I use in all my python projects,
Function to create a customized logger object.
All of my projects have benefitted due to simple logging. The impact of a few minutes of simple scripting is nearly tangible.
I have used log files for a variety of reasons. Some of them are,
- Communication — to share key information.
- Visibility — messages from modules reveal their participation.
- Journaling — keeping a historical record of events.
- Simplification — use simple texts to represent complex code.
- Analytics — log timestamps indicate performance, etc.
Going back to the room analogy, with the right logging setup, we can create a room filled with talented people creating something beautiful.
