What Is the Outbox Pattern?

Quick answer
The outbox pattern writes the event as a row in the same transaction as the business data, then a separate process delivers it from that table.
It exists to solve the dual-write problem: you cannot atomically update the database and publish to a broker, so one of the two can always be lost.
Because the row and the data commit together, the event is guaranteed to exist if the change happened; delivery becomes at-least-once.
A relay process polls or tails the outbox table and publishes each row, marking it delivered.
Consumers must be idempotent, because at-least-once means a duplicate is normal rather than exceptional.

What is the outbox pattern?

It is a technique for reliably publishing a message that must correspond to a database change. Instead of writing the data and then sending the message, the application writes the data and inserts a row describing the message into an outbox table, both inside the same transaction. A separate process reads that table and publishes the messages. The message now has the same durability as the data, because it was part of the same commit.

The problem it solves is called the dual write: two independent systems, the database and the broker, cannot be updated atomically. Whichever you do first, a crash in between leaves an inconsistency — data committed with no message, or a message for data that was rolled back. The outbox collapses the two writes into one by making the message part of the database’s own transaction.

Why ordering the two writes does not work

Writing the data first and publishing second loses the message whenever the process dies after the commit and before the publish. Writing and publishing first can announce a change that the transaction then rolls back, which is worse, because consumers act on something that never happened. Retrying the publish does not help: the failure is between the two systems, not inside either one.

Wrapping the publish in the transaction does not help either, because the broker is not transactional with the database. Distributed transactions across both would, and that reintroduces exactly the coordination cost the outbox exists to avoid. So the pattern is the standard answer: one transactional system, with delivery handled afterwards.

How the relay works

The relay is a process that reads undelivered rows in order, publishes each to the destination, and marks it delivered. It can poll on an interval, which is simple, or tail the table’s change log, which is lower-latency. Two details determine whether it is correct in practice. It must publish before marking, so a crash between the two at worst causes a duplicate rather than a loss. And it must be able to catch up after downtime, which means ordering and backfill rather than jumping to the newest row.

Because the relay can be interrupted at the worst moment, delivery is at-least-once: every message arrives, and some arrive twice. That is the price of solving the dual write without distributed transactions, and it is a much better price than silently losing events. It pushes the idempotency requirement downstream, to every consumer, where it is a design constraint rather than an afterthought.

-- The outbox table lives in the same database as the data.
CREATE TABLE outbox (
  id          bigserial PRIMARY KEY,
  event_type  text        NOT NULL,
  payload     jsonb       NOT NULL,
  created_at  timestamptz NOT NULL DEFAULT now(),
  published_at timestamptz
);

-- The business change and the event commit together.
BEGIN;
UPDATE orders SET status = 'paid' WHERE id = 42;
INSERT INTO outbox (event_type, payload)
VALUES ('order.paid', jsonb_build_object('order_id', 42));
COMMIT;

-- The relay claims a batch, publishes it, then marks it delivered.
-- Claim first with SKIP LOCKED so two relays do not take the same row.
SELECT id, event_type, payload FROM outbox
WHERE published_at IS NULL
ORDER BY id
LIMIT 100
FOR UPDATE SKIP LOCKED;

Costs and when not to bother

The costs are real. There is an extra table and its growth, a relay process and its operations, added latency between the change and the event, and the requirement that consumers tolerate duplicates. For a small application with one consumer that reads the database directly, none of it is necessary. The pattern earns its place when a change must be communicated to another system that cannot see your database, and losing that message would be a real problem.

A useful test: does anything outside this database need to know, and would it be wrong if it never found out? If the answer is yes, the outbox is the reliable way to say so. If the answer is that the consumer can simply read the data on its own schedule, you want a scheduled job, not an event pipeline, and the outbox is complexity without a purpose.

Where this bites vibecoders
Generated code publishes events with a call to the broker right after a database write, which is the dual-write bug in its most common form: it works in every test and loses messages only when a process dies at the wrong moment. The failure is invisible until the day an integration mysteriously misses an update. If an assistant proposes “update the row, then publish”, the question to ask is what happens if the process stops between those two lines.

Where AI coding assistants get this wrong

Sending the message immediately after the commit, outside the transaction.
Publishing before the commit, announcing changes that may roll back.
Marking the outbox row delivered before publishing it.
Assuming exactly-once delivery and writing consumers that cannot handle a duplicate.
Letting the outbox table grow without a retention or cleanup policy.
Adopting the pattern where a direct read would have been sufficient.

Checklist

Confirm an external system genuinely needs the event.
Write the event row inside the same transaction as the data change.
Relay with a claim-then-publish-then-mark order.
Use row locking with SKIP LOCKED so relays do not collide.
Make every consumer idempotent, keyed on an event id.
Set a retention policy for delivered rows.

FAQ

What is the dual-write problem?

It is the impossibility of updating a database and publishing to a message broker atomically when they are separate systems. Whichever you do first, a crash in between leaves the two inconsistent. The outbox removes the second write from the critical path by making the message a row in the same transaction, so there is only one system to commit.

Does the outbox give exactly-once delivery?

No. It gives at-least-once, because the relay can crash after publishing and before marking the row, in which case it republishes on restart. Exactly-once across systems generally requires cooperation on both sides and is not free. Designing consumers to be idempotent is the practical answer and is required in any case.

How does the relay avoid sending the same event twice?

It cannot entirely, and it should not pretend to. It can reduce duplicates by claiming a batch with row locks, publishing, then marking, and by ordering by id. Some duplicates remain possible after a crash. Consumers deduplicate on the event id, which is why every event needs a stable identifier.

Should the outbox be in the same transaction as the business data?

Yes — that is the entire point. If the insert into the outbox is committed separately, you are back to two writes and the same failure window, just moved. The atomicity of the data change and the event record is what makes the pattern work, so both must be inside one transaction.