How to Run PostgreSQL on k3s (With Backups That Actually Restore)

Quick answer
Run Postgres as a StatefulSet with a volume claim template, not a Deployment with an emptyDir — a Deployment reschedules the pod and loses the database.
k3s’s bundled local-path provisioner binds storage to the node’s disk, so the data lives on that one server and dies with it unless you back it up elsewhere.
Keep the password in a Secret, expose the database only with a ClusterIP Service, and never publish 5432 to the internet.
A backup job that has never been restored is a hypothesis; run an actual restore into a scratch database before you trust it.
A single Postgres instance on one node has no failover. Know what losing the node costs you before you put production data on it.

Before you start

You need a working k3s cluster with kubectl access, a container registry or the official Postgres image, and somewhere off the node to put backups — object storage, another server, or anything that is not the disk you are backing up. Decide the database size and set the volume request to match, because the local-path provisioner creates a directory on the node and will not grow it for you. And accept the architecture up front: this is one Postgres, on one machine, with no replica. That is perfectly reasonable for a side project and wrong for anything where an hour of downtime or a few hours of lost writes is unacceptable.

Step 1 — Store the password in a Secret

Create the credential as a Secret rather than an environment variable in the manifest, and generate it rather than inventing it. Secrets are not encrypted at rest by default, so treat the cluster as a place that can read this value — see the guide on what Kubernetes Secrets actually protect — and keep the source of truth somewhere you can rotate.

kubectl create secret generic postgres-auth \
  --from-literal=POSTGRES_PASSWORD="$(openssl rand -base64 24)" \
  --from-literal=POSTGRES_USER=app \
  --from-literal=POSTGRES_DB=app

Step 2 — Declare the PersistentVolumeClaim

On k3s, a PVC with no storageClassName binds to the bundled local-path provisioner, which creates a directory under /var/lib/rancher/k3s/storage on the node. That is enough for a StatefulSet to survive pod restarts and reschedules on the same node. When the StatefulSet creates its own claim through a volume claim template you can skip this step, but declaring it explicitly documents how much disk the database is allowed to take.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pgdata
spec:
  accessModes: [ReadWriteOnce]
  resources:
    requests:
      storage: 20Gi

Step 3 — Deploy Postgres as a StatefulSet

Use a StatefulSet so the pod keeps a stable identity and its own claim. Set resource requests and limits, because an unbounded Postgres will consume the node’s memory before it starts refusing queries, and wire the readiness probe to pg_isready so Kubernetes does not route traffic to a database that is still recovering. The volume claim template keeps the data volume attached to this pod’s identity across restarts.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
spec:
  serviceName: postgres
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - name: postgres
          image: postgres:16.4
          ports:
            - containerPort: 5432
          envFrom:
            - secretRef:
                name: postgres-auth
          env:
            - name: PGDATA
              value: /var/lib/postgresql/data/pgdata
          resources:
            requests:
              cpu: 250m
              memory: 512Mi
            limits:
              memory: 1Gi
          readinessProbe:
            exec:
              command: [pg_isready, -U, app]
            initialDelaySeconds: 10
            periodSeconds: 10
          volumeMounts:
            - name: pgdata
              mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
    - metadata:
        name: pgdata
      spec:
        accessModes: [ReadWriteOnce]
        resources:
          requests:
            storage: 20Gi
---
apiVersion: v1
kind: Service
metadata:
  name: postgres
spec:
  selector:
    app: postgres
  ports:
    - port: 5432
      targetPort: 5432

Step 4 — Connect your app without exposing the database

The Service above is a ClusterIP, which is what you want: reachable from inside the cluster and nowhere else. Your app points at the hostname postgres on port 5432 and reads the password from the same Secret. If you need to inspect the database from your workstation, port-forward rather than publishing the port — kubectl port-forward svc/postgres 5432:5432 opens a tunnel through the API server, whereas a LoadBalancer or NodePort service puts an open Postgres on the internet, where automated scanners find it within hours.

# Local access without exposing the database
kubectl port-forward svc/postgres 5432:5432
# then connect from your machine
psql 'postgresql://app@localhost:5432/app'

Step 5 — Back it up off the node and restore it once

A CronJob running pg_dump and uploading the result off the node is the minimum viable backup. Schedule it for a low-traffic window, keep more than one generation, and make the job fail loudly if the upload does not succeed — a backup job that silently errors every night is worse than no backup, because you have stopped worrying. Then do the part almost everyone skips: restore the dump into a scratch database and query it. Until you have done that, you have a file, not a recovery plan.

apiVersion: batch/v1
kind: CronJob
metadata:
  name: postgres-backup
spec:
  schedule: '17 3 * * *'
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: OnFailure
          containers:
            - name: dump
              image: postgres:16.4
              envFrom:
                - secretRef:
                    name: postgres-auth
              command:
                - sh
                - -c
                - >-
                  pg_dump -h postgres -U app -Fc app > /tmp/app.dump &&
                  echo 'upload /tmp/app.dump to off-node storage here'

Where this bites vibecoders
Ask an assistant for ‘a Postgres database on my cluster’ and you will get a Deployment with an emptyDir or no volume at all, credentials written into the env block, no resource limits, and a Service of type LoadBalancer with port 5432 published to the internet. It will apply cleanly and work until the pod is rescheduled, at which point the database is empty. The three fixes that matter are small — StatefulSet with a volume claim template, a Secret for the password, a ClusterIP Service — and the fourth, an off-node backup you have actually restored, is the one no generated manifest ever includes.

Where AI coding assistants get this wrong

Using a Deployment with an emptyDir volume, so the database is lost on every reschedule.
Publishing Postgres with a LoadBalancer or NodePort Service instead of a ClusterIP.
Writing the password inline in the manifest instead of a Secret.
Omitting resource limits, letting Postgres consume the whole node.
Reading PGDATA on a mount that already has a lost+found directory, which makes Postgres refuse to start.
Assuming the backup job succeeded because the CronJob exists.

Checklist

Deploy Postgres as a StatefulSet with a volume claim template.
Keep credentials in a Secret and rotate them on a schedule.
Expose the database with a ClusterIP Service only.
Set memory requests and limits so the node cannot be consumed.
Point the readiness probe at pg_isready.
Schedule an off-node backup that fails loudly when it does not upload.
Restore a backup into a scratch database and query it.
Decide what happens to the app when this single node is gone.

FAQ

Should I run Postgres in Kubernetes at all?

For a small single-node app it is a reasonable trade: you get the same deployment workflow as the rest of your stack. The counter-argument is that a managed database gives you point-in-time recovery, failover, and automatic upgrades for a modest monthly cost, and those are exactly the things a single-node StatefulSet does not provide. If the data matters and the budget allows, a managed database is usually the better call.

Why does my Postgres pod crash with a permission or initdb error?

The usual cause on Kubernetes is mounting a volume at /var/lib/postgresql/data that already contains files — a lost+found directory from the underlying filesystem is the classic example. Set PGDATA to a subdirectory such as /var/lib/postgresql/data/pgdata so the database initializes into an empty directory.

How do I upgrade Postgres on k3s?

Dump the database, then change the image tag to the new major version and restore into it, because a major version bump changes the on-disk format. Minor version upgrades can be done in place by changing the tag. Either way, verify the restore before deleting the old volume, and keep the previous volume until you are satisfied.