What Is Log Rotation (and Why Do Your Logs Keep Disappearing)?

Quick answer
Log rotation renames and archives log files on a schedule so they don’t grow until they fill the disk.
Without it, an app that logs anything at all eventually fills the disk and the whole server crashes.
Services like logrotate handle it automatically; managed log platforms (Sentry, Better Stack) replace local files entirely.

Why do logs fill up the disk?

An app that logs a line per request produces megabytes a day; one that logs per line of debug output produces gigabytes. Log files only grow. Without rotation, the file expands until the disk is full, and a full disk breaks far more than logging: databases stop writing, the OS becomes unstable, and backups fail. It’s a slow-motion outage that starts with ‘the logs are huge’ and ends with ‘the server is down’.

How does log rotation work?

A tool like logrotate runs on a schedule — daily is typical — and applies rules per log file: rename the current file with a date suffix (app.log becomes app.log.1), compress old ones, and delete files older than a retention window (keep 7 files, or 30 days). The app keeps writing to the same filename, so it never notices. A typical config keeps seven daily files, compressed, with an empty file left in place so the app’s file handle keeps working.

# /etc/logrotate.d/myapp
/var/log/myapp/*.log {
    daily
    rotate 7
    compress
    missingok
    notifempty
    copytruncate
}

What’s the modern alternative to rotating local logs?

Most deployed apps shouldn’t manage local log files at all: write logs to stdout, let the platform capture them, and use a log management service (Sentry, Better Stack, Datadog) for search and retention. The platform handles rotation and storage, and you get search, alerting, and retention policies for free. Log rotation remains essential for self-hosted apps and servers that write to files directly.

Where this bites vibecoders
AI assistants tell you to ‘add logging’ for debugging but never ‘add log rotation’, so the first production incident is often the disk filling up from the very logs that were supposed to help. The fix is a three-line logrotate config or, better, stdout logging plus a managed log service. It’s a classic invisible-operations detail: the assistant solves the debugging problem and accidentally creates a disk-full problem.

Where AI coding assistants get this wrong

Adding verbose logging everywhere with no rotation or retention, guaranteeing a full disk.
Writing logs to files the platform never sees, so production debugging is blind.
Rotation configs with no compression, so ‘rotated’ files still eat the disk.
Deleting the active log file instead of rotating it, so the app keeps writing to a deleted inode and nothing logs anymore.

Checklist

Configure rotation for every self-hosted log file: daily, compressed, bounded retention.
Prefer stdout logging plus a managed log service on platforms and containers.
Monitor disk usage so a log growth problem surfaces before the disk is full.
Verify rotation works by checking that old files are compressed and pruned.

FAQ

Why did my logs suddenly stop appearing?

Classic cause: the app opened the log file, someone deleted or rotated it out from under it, and the app keeps writing to the deleted file’s inode. The file is growing on disk with no name. Restart the app or configure copytruncate-style rotation that leaves the file in place.

How long should I keep logs?

Enough to debug the incidents you actually get: 7-30 days locally is typical, and more if compliance requires it. Keep them long enough to investigate last week’s outage, short enough that disk and costs stay sane. Managed log services make long retention cheap and searchable.