Logging in Go: Tips and Tricks for Effective Log Analytics

What connects Docker, Kubernetes, and Prometheus? All these cloud-native technologies have been developed using the Go programming language, also known as Golang. Known for its minimalism and efficiency, Go offers advanced features like channels and goroutines for concurrent system-level programming. 

This blog delves into various Go logging packages and highlights their unique advantages. We will explore how to use Observe in Go environments to gather, centralize, and manage logs effectively.

Logs are the foundational pillar in developing Observability for applications. They provide granular insights into the system’s internal state and behavior over time by recording events and data points within an application, offering developers and operations teams a detailed, timestamped chronology of operations, errors, and interactions. This data is invaluable for troubleshooting issues, understanding performance bottlenecks, and monitoring the application’s health in real-time. By analyzing logs, teams can quickly identify patterns, predict potential problems before they become critical, and make informed decisions to enhance system reliability and performance.

Adopting specific strategies and best practices can help you pinpoint and address performance issues when developing Go applications. 

The Most Straightforward Way to Begin

Several choices are available when selecting a logging package in Go. The most straightforward approach is to use the native logging library, log, which is part of the Go runtime. It’s easily implemented, readily accessible, and automatically provides useful timestamps without additional setup. Let’s begin with a basic example:

package main

import (
  "log"
)

func main() {
  log.Print("Basic Log Message in Go!!")
}

When you compile and execute the program, it will produce the following log message:

$ 2024/04/28 11:04:03 Basic Log Message in Go!!

While it may be better than having no logging, essentially, it is just an unstructured string paired with a timestamp. Print and Println functions are the most frequently used in the log package, yet log does not support various levels natively; additional steps are required. However, the package does include functions that provide more than basic logging. For instance, log.Fatal logs a message at a Fatal level and then terminates the process. Similarly, log.Panic logs a message at a Panic level before triggering a call to panic(), a built-in Go function.

By default, the log package outputs to stderr. However, if you need to direct your logs to a different location, you can use the log.SetOutput function. This function enables you to designate a custom io.Writer as the output destination, such as a file:

 

package main

import (
	"log"
	"os"
)

func main() {
	file, err := os.OpenFile("info.log", os.O_CREATE|os.O_APPEND
                     |os.O_WRONLY, 0644)
	if err != nil {
		log.Fatal(err)
	}

	defer file.Close()

	log.SetOutput(file)
	log.Print("Logging to a file using Go!")
}

Compiling and executing the program generates a file named logfile.log, where messages are continuously written and appended.

The log package also offers the ability to enhance logs with contextual details like file names, line numbers, dates, and times through Log Flags.

Structured Logging in Go

For many years, Golang lacked native support for structured logs, forcing developers to choose one of the many frameworks and libraries that support structured logs. We will discuss these libraries next; however, starting with Golang 1.21, structured logging is supported natively with ‘slog.’ Unlike traditional logging methods that produce plain text, slog leverages structured data formats, typically JSON, to create human-readable and machine-friendly logs. This approach facilitates more efficient parsing and processing of log data, which is particularly advantageous in environments where logs are aggregated and analyzed by automated systems. slog also supports contextual logging, where logs can include additional, dynamically generated data fields that provide deeper insight into the application’s behavior. Structured logs can significantly enhance debugging and monitoring capabilities, making slog an excellent choice for developers looking to implement robust logging practices in their Go applications.

slog comes with two built-in handlers. A TextHandler and a JSONHandler.

package main

import (

   "log/slog"

   "os"

)

func main() {

   logger := slog.New(slog.NewJSONHandler(os.Stdout, nil))

   logger.Info("hello, world from", "user", os.Getenv("USER"))

}

When this is compiled, you will get:

{"time": "2024-04-28T15:11:45.219746-07:00", "level": "INFO", "msg": 
"hello, world from", "user": "amits"}

{"time":"2024-04-28T15:11:45.219746-07:00","level":"INFO","msg":
"hello, world from","user":"amits"}

You’re not restricted to using only the pre-defined handlers. You have the option to create your own handler by implementing the slog.Handler interface. A custom handler can either produce output in a specific format or enhance another handler by adding new functionalities. For instance, the slog documentation includes an example of how to develop a wrapping handler that adjusts the minimum log level required for messages to be shown.

Logging Packages in Go

You may have chosen other log frameworks in Go for performance or following conventions. The two popular ones are Logrus and Zap.

Logrus is a structured logging package optimized for JSON output. It simplifies the parsing and interpretation of your Go application logs. Currently, Logrus is in maintenance mode, meaning it no longer receives new features but continues to receive support for security updates, backward compatibility, and performance improvements.

With Logrus, you can enhance your JSON logs by including standard fields using the WithFields function. The package provides seven levels of logging: Trace, Debug, Info, Warn, Error, Fatal, and Panic, allowing you to log messages at the appropriate severity.

These log levels add more context to your messages by categorizing them and prove useful when you want to limit the amount of operational data you’re analyzing.

Zap

Zap is a logger that outputs in JSON format, engineered to minimize memory allocation and reduce reliance on reflection and string formatting. It is acclaimed for its high speed and low memory usage, which make it particularly well-suited for scaling applications.

Zap offers two main logging APIs: zap.Logger and zap.SugaredLogger. The former is ideal for high-performance scenarios but only supports structured logging. In contrast, the latter has a more developer-friendly syntax.

Just how swift is Zap? It’s exceptionally fast! Benchmarks show that Zap operates approximately 1677% faster than Logrus when logging messages that include ten contextual fields! It is 188% faster than slog.

Troubleshooting in real-time? You’ll love Observe

Observe is a modern, cloud-native observability platform that delivers fast performance and petabyte scalability while minimizing cost. 

Observe provides complete flexibility in data collection, including log data ingestion. You can ingest your log data most commonly via the Host Monitoring App, OpenTelemetry File Receiver, HTTP ingest APIs, or the Kubernetes App in containerized environments. Once ingested, you can analyze the log data in Log Explorer, and Live mode reduces the end-to-end latency of new data arriving at Observe and showing up on your screen to mere seconds.

If you are using the native log package or Zap’s SugaredLogger and want the searchability of structured logs, you can easily parse the text-based log messages in Observe after ingestion to extract key-value pairs, achieving the best of both worlds–developer-friendly instrumentation and fast log analytics.

A carefully crafted logging strategy can provide valuable insights into your Go applications. Many approaches and frameworks in Go can help achieve effective logging. Select the library that best meets your needs, and streamline the logging process by centralizing logs across your stack using Observe.

We invite you to try Observe for 14 days for free and start analyzing log data from your Go applications.