ZFS Log Compression on FreeBSD

System logs are an important source of information about events that happened during the runtime of the system. FreeBSD’s default logging system, called syslog, writes them to /var/log/ as different files based on the subject matter. The most common events, as well as those that don’t fit anywhere else, are written to /var/log/messages.

Over time, logs grow and get rotated by the newsyslog daemon. During the rotation, newsyslog creates a new log and renames the old log file, while compressing it to save space. The default naming scheme is to add a number (or increase it) based on the age. For example, the current log file gets the extension .1 added and newsyslog renames the old .1 file to .2. Log files that would rotate past .7 get deleted as the oldest logs. Two configuration files control the size, total number of logs, and other rotation rules and how to log in general: /etc/newsyslog.conf and syslog.conf.

OpenZFS has some powerful and efficient compression algorithms available to compress data on the fly in a dataset. Whenever the system writes a log file, ZFS compresses it in main memory before it reaches the underlying storage. When reading the file again, ZFS transparently uncompresses the requested data and presents it uncompressed to the application. This yields tremendous benefits both in compression ratio and convenience, since the filesystem does all the compression steps automatically. The syslog compression is still done in the traditional way by zipping and unzipping files. When accessing the contents of an old log file, the user needs to uncompress or view it using a program like bzcat first. ZFS also yields higher compression ratios, which is why it makes sense to let ZFS take control of the syslog compression, too.

When running a root-on-ZFS system /var/log resides on a separate dataset. Display the current compression algorithm used by running the following command:

We use the default pool name (zroot) used in the FreeBSD installer. Change it in these examples when zpool list -Ho name returns a different name. If this returns off, then set it to one of the available compression algorithms listed in zfsprops. Good starting values are lz4 or even zstd.

Remember that ZFS compresses new data, not the existing files that are in the dataset already. We’ll make some changes to disable newsyslog’s compression next. FreeBSD documents the fields of /etc/newsyslog.conf in newsyslog.conf(5). The last column (flags) in the configuration file contains the compression algorithm to apply when rotating a log file. Since we want ZFS to do the compression, remove the J from this column for each such log file listed there. Once finished with the changes, save and exit newsyslog.conf.

This step is not necessary, but provides a simple way of identifying the timeframe from a rotated log file. We wrote above that older log files get an extension from .1 up to .7. When looking for an older log, it may be tedious to open all those old log files to find a specific one from a certain date. Wouldn’t it be easier to have the file’s rotation date used instead? For example, messages.20240914T210000 indicates when the messages file was last rotated: September 14, 2024, at 21:00:00. Of course, the format is configurable and described in the -t parameter to newsyslog.

We can apply this naming scheme to each log file created by editing /etc/crontab. A line with a comment above like this is what we need to find:

Change it to the following:

Save and exit.

To let these changes take effect, restart both the newsyslog and syslog daemons:

ZFS will now see uncompressed log files in /var/log. Log rotation still happens, but the old logs are not compressed by newsyslog anymore. To check the space savings achieved by the compression, run this command:

Here is the output from a system where this change happened weeks ago:

This means the log is 1/40 the original, uncompressed size. ZFS also provides details about the original size without compression as a property:

The result is a good amount of space saved for a small change. The new log files also receive the new naming scheme when rotating from now on. Find out more about logging in FreeBSD by reading the Configuring System Logging handbook chapter.