How to rotate and compress old logs on Ubuntu 24.04

Configure the logrotate utility to automatically compress and rotate system and application logs, preventing disk space issues on Ubuntu 24.04 servers. This guide targets Ubuntu 24.04 with logrotate 3.19.0 and requires root privileges. You will set up a cron job to trigger the rotation at the correct interval and customize the compression settings.

Prerequisites

• Ubuntu 24.04 LTS operating system installed.
• Root access or sudo privileges.
• logrotate 3.19.0 package installed (default on Ubuntu 24.04).
• At least 500MB of free disk space for temporary compression files.

Step 1: Check the current logrotate configuration

Verify that the logrotate service is installed and check the default configuration file location. You will find the main configuration file at /etc/logrotate.conf. Run the following command to see the default settings and confirm the compression is enabled by default.

sudo cat /etc/logrotate.conf

# Example output snippet:
# Compress
# Create new (rotated) log file after rotating old one
# Keep the last 4 rotated logs by default
# Run logrotate every 14 days by default

Step 2: Create a custom logrotate configuration for specific logs

Edit the main configuration file to define custom rules for specific log directories or applications. Add a new block for your application logs or modify the existing /var/log section. Use sudo nano to open the file and append your custom rules at the end.

sudo nano /etc/logrotate.d/custom-apps

Enter the following configuration to rotate logs older than 7 days and compress them. This example targets logs in /var/log/myapp. Adjust the path and log names to match your specific environment.

/var/log/myapp/*.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    create 0640 root adm
    postrotate
        systemctl reload myapp.service > /dev/null 2>&1 || true
    endscript
}

Step 3: Configure system-wide log compression policy

Modify the global settings to ensure all logs are compressed automatically after rotation. Open the main configuration file and ensure the compress directive is present. If you want to change the default retention period for all logs, update the rotate directive in /etc/logrotate.conf.

sudo nano /etc/logrotate.conf

Add or verify the following lines in the file to enforce compression globally:

compress
compresscmd /bin/gzip
uncompresscmd /bin/gunzip
compressext .gz

Set the retention count to keep 14 rotated logs by default, which balances disk usage and history. This prevents the disk from filling up with uncompressed text files.

Step 4: Set up the cron job for manual triggering

Although logrotate runs via /etc/cron.d/logrotate, you can create a specific cron job to run your custom rules. Open the crontab for root and add a line to run the rotation daily at 3:00 AM. This ensures the server is less active during the rotation process.

sudo crontab -e

Add the following line to the file to trigger the rotation script daily:

0 3 * * * /usr/sbin/logrotate -f /etc/logrotate.conf

The -f flag forces rotation even if the logs have not reached the size limit. This is useful for testing or ensuring logs are rotated during maintenance windows.

Step 5: Test the configuration syntax

Before applying the changes, validate the configuration to ensure there are no syntax errors. Run the logrotate command with the debug and verbose flags to see what would be rotated without actually rotating the files.

sudo logrotate -d /etc/logrotate.conf

You will see output listing the files that would be rotated and the actions taken. Check for any error messages regarding file permissions or missing directories. If errors appear, correct the file paths or permissions before proceeding.

Verify the installation

Run the logrotate command with the force flag to perform an immediate rotation and compression. Check the /var/log/myapp directory to confirm that the old logs are compressed with the .gz extension.

sudo logrotate -f /etc/logrotate.conf

Verify the compression by listing the directory contents and checking the file sizes. The rotated files should be significantly smaller than the original logs. Ensure the new log files have the correct permissions set by the create directive.

Troubleshooting

If the rotation fails, check the system logs for error messages. The primary log for logrotate is /var/log/syslog or /var/log/logrotate. Look for permission denied errors or paths that do not exist.

sudo grep logrotate /var/log/syslog

Common issues include incorrect file permissions on the log directory. Ensure the directory is readable by the user running logrotate. If the postrotate script fails, it is often because the service does not exist or the reload command is incorrect. Modify the script to match your specific service name.

If logs are not being compressed, verify that the compress directive is active in the configuration file. Ensure the compresscmd is set to /bin/gzip. If the compression is slow, consider increasing the available disk space or reducing the number of logs retained.

To disable rotation for a specific log file, add a nocreate directive or comment out the block in the configuration file. This prevents accidental overwriting of critical logs that require manual management.

Always test your configuration changes in a staging environment before deploying them to production servers. A failed rotation can lead to disk full errors if the old logs are not removed correctly. Monitor the disk usage after the first few rotation cycles to ensure the retention policy is working as expected.