Sign in Subscribe
By Ken P Griesemer in Coding —

Structured Logging with slog in Go

In the Go programming language, the introduction of the log/slog package in Go 1.21 has brought structured logging to the standard library.

Structured Logging with slog in Go

Introduction

Structured logging has become an essential practice in modern software development, providing developers with a powerful tool to gain deeper insights into their applications. In the Go programming language, the introduction of the log/slog package in Go 1.21 has brought structured logging to the standard library, making it more accessible to Go developers.

The log/slog package, often referred to as simply "slog", is the result of a proposal by Jonathan Amsterdam, a member of the Go team. The package aims to address the shortcomings of the existing log package in Go, which has been limited to basic, unstructured logging. With slog, Go developers can now enjoy the benefits of structured logging, including improved log parsing, filtering, and analysis.

In this comprehensive article, we will explore the architecture and features of the slog package, providing you with a thorough understanding of how to leverage it in your Go projects. We'll cover topics such as initialization, log levels, structured data, and custom handlers, as well as best practices and advanced techniques for optimizing your logging infrastructure.

Understanding the Slog Architecture

At the core of the slog package are three main components: the Logger, the Record, and the Handler.

The Logger

The Logger is the user-facing API that developers interact with to create log entries. It provides a set of methods, such as Info(), Debug(), Warn(), and Error(), which allow you to log messages at different severity levels.

When you call one of these methods, the Logger creates a Record object that encapsulates the log entry's details, including the timestamp, log level, message, and any additional structured data.

The Record

The Record is a representation of a single log entry. It contains the following standard fields:

  • Time: the timestamp of the log entry
  • Level: the log level (e.g., Info, Debug, Warn, Error)
  • Message: the log message

In addition to these standard fields, you can attach arbitrary key-value pairs and other metadata to the Record, such as the calling function, source file, and line number.

The Handler

The Handler is responsible for processing the log entries and determining their final destination. Slog comes with two built-in Handlers: TextHandler and JSONHandler. The TextHandler produces log messages in a human-readable, key-value format, while the JSONHandler outputs logs in a structured JSON format.

Developers can also create their own custom Handlers by implementing the Handler interface. This allows for greater flexibility in how logs are formatted and where they are sent, such as writing logs to a file, a network endpoint, or a centralized logging service.

Initializing Slog

To get started with slog, you need to create a Logger instance and configure it with a Handler. Here's an example of initializing a Logger with the TextHandler that writes logs to stdout:

package main

import (
    "os"
    "log/slog"
)

func main() {
    logger := slog.New(slog.NewTextHandler(os.Stdout))
    logger.Info("Hello, slog!")
}

In this example, we create a new Logger instance using the slog.New() function and pass it a TextHandler that writes to the standard output (os.Stdout). We then use the Info() method to log a message at the Info level.

The output of this program will be:

2023-04-10T02:10:00.000Z INFO Hello, slog!

You can also customize the Logger by setting options, such as the default log level or the timestamp format:

logger := slog.New(
    slog.NewTextHandler(os.Stdout),
    slog.WithLevel(slog.LevelDebug),
    slog.WithTimeFormat("2006-01-02 15:04:05"),
)

In this example, we set the default log level to Debug and the timestamp format to a more human-readable format.

Logging with Slog

Slog provides several methods for logging messages at different severity levels:

  • Debug(msg string, ...Attr): logs a message at the Debug level
  • Info(msg string, ...Attr): logs a message at the Info level
  • Warn(msg string, ...Attr): logs a message at the Warn level
  • Error(msg string, ...Attr): logs a message at the Error level

The Attr parameter is used to attach structured data to the log entry. You can pass key-value pairs, like this:

logger.Info("User logged in", "user_id", 123, "ip_address", "192.168.1.100")

This will produce the following output with the TextHandler:

2023-04-10T02:10:00.000Z INFO User logged in user_id=123 ip_address=192.168.1.100

Alternatively, you can use the With() method to attach a set of attributes to the Logger, which will then be included in all subsequent log entries:

logger := logger.With("app_name", "my-app", "version", "1.2.3")
logger.Info("User logged in", "user_id", 123)

This will produce the following output:

2023-04-10T02:10:00.000Z INFO User logged in app_name=my-app version=1.2.3 user_id=123

Structured Data and Attributes

One of the key features of slog is its support for structured data. In addition to the standard log fields (timestamp, level, and message), you can attach arbitrary key-value pairs and other metadata to the log entries.

These additional data points are called "attributes" in slog terminology. Attributes can be of various types, such as strings, integers, booleans, and even complex data structures.

Here's an example of logging a structured error with additional context:

import (
    "errors"
    "log/slog"
)

func main() {
    logger := slog.New(slog.NewTextHandler(os.Stdout))

    err := errors.New("database connection failed")
    logger.Error("Failed to connect to database", "error", err, "host", "db.example.com", "port", 5432)
}

This will produce the following output:

2023-04-10T02:10:00.000Z ERROR Failed to connect to database error="database connection failed" host=db.example.com port=5432

In this example, we've attached the error object, the database host, and the database port as attributes to the log entry. This additional context can be invaluable when debugging issues in production.

Custom Handlers

While slog comes with the TextHandler and JSONHandler out of the box, you may want to create your own custom Handlers to suit your specific logging requirements. This could include writing logs to a file, a network endpoint, or integrating with a third-party logging service.

To create a custom Handler, you need to implement the Handler interface, which has a single method:

type Handler interface {
    Handle(Record) error
}

The Handle() method is responsible for processing the Record object and sending the log entry to its final destination.

Here's an example of a custom Handler that writes logs to a file:

import (
    "os"
    "log/slog"
)

type FileHandler struct {
    file *os.File
}

func NewFileHandler(filename string) (*FileHandler, error) {
    file, err := os.OpenFile(filename, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0644)
    if err != nil {
        return nil, err
    }
    return &FileHandler{file: file}, nil
}

func (h *FileHandler) Handle(r slog.Record) error {
    _, err := fmt.Fprintf(h.file, "%s\n", r.String())
    return err
}

In this example, we create a FileHandler struct that wraps an os.File object. The NewFileHandler() function creates a new file handler, opening the file in append mode. The Handle() method writes the log entry to the file using the Record.String() method, which returns a human-readable string representation of the log entry.

You can then use this custom Handler when creating your Logger:

fileHandler, err := NewFileHandler("app.log")
if err != nil {
    // handle error
}

logger := slog.New(fileHandler)
logger.Info("This log entry will be written to the app.log file")

Best Practices and Advanced Techniques

As you integrate slog into your Go projects, there are several best practices and advanced techniques you should consider:

Consistent Logging Conventions

Establish a set of conventions for your logging practices, such as:

  • Consistent key naming (e.g., snake_case, camelCase)
  • Appropriate use of log levels (e.g., Debug, Info, Warn, Error)
  • Inclusion of relevant context (e.g., request IDs, user IDs, error details)

You can enforce these conventions using a linter like sloglint, which can help ensure consistency across your codebase.

Centralized Logging Infrastructure

While writing logs to local files is a good starting point, it's often beneficial to centralize your logging infrastructure. This can involve integrating with a log management system, such as Elasticsearch, Splunk, or Datadog, which can provide advanced search, analysis, and alerting capabilities.

When centralizing your logs, it's important to decouple the task of writing logs from shipping them to the centralized system. This can be achieved by first writing logs to local files and then forwarding them to the centralized system asynchronously. This approach ensures that your application can continue to function even if the centralized logging system or network connection is unavailable.

Performance Optimization

Logging can have a non-trivial impact on application performance, especially in high-throughput scenarios. To optimize the performance of your logging infrastructure, consider the following techniques:

  • Asynchronous Logging: Use a separate goroutine or a buffered channel to handle log writes, allowing your application to continue processing requests without waiting for logs to be written.
  • Structured Data Optimization: Carefully consider the structured data you attach to log entries, as excessive or unnecessary data can impact performance. Use only the most relevant attributes.
  • Custom Handlers: Implement custom Handlers that are optimized for your specific use case, potentially leveraging low-level libraries or techniques to minimize overhead.
  • Conditional Logging: Use the slog.ShouldLog() function to conditionally log messages based on the current log level, avoiding unnecessary work when logging is disabled or set to a higher level.

By following these best practices and advanced techniques, you can ensure that your logging infrastructure is robust, efficient, and tailored to the needs of your Go applications.

Conclusion

The introduction of the log/slog package in Go 1.21 has brought structured logging to the standard library, making it more accessible to Go developers. By understanding the architecture of slog, including the Logger, Record, and Handler components, you can effectively leverage this powerful tool to improve the observability and debuggability of your Go applications.

Through the examples and best practices covered in this article, you should now have a solid foundation for integrating slog into your Go projects. Remember to establish consistent logging conventions, centralize your logging infrastructure, and optimize your logging performance to ensure that your application's logging capabilities are robust and efficient.

As the Go ecosystem continues to evolve, the slog package is poised to become an essential part of the standard toolset for Go developers, providing a seamless and powerful way to incorporate structured logging into their applications.

Citations:
[1] https://dusted.codes/creating-a-pretty-console-logger-using-gos-slog-package
[2] https://go.dev/blog/slog
[3] https://mrkaran.dev/posts/structured-logging-in-go-with-slog/
[4] https://www.reddit.com/r/golang/comments/16dz2t3/slog_write_to_terminal_and_file/
[5] https://betterstack.com/community/guides/logging/logging-in-go/

Subscribe to TheBuggerUs

Don’t miss out on the latest issues. Sign up now to get access to the library of members-only issues.
jamie@example.com
Subscribe