How to Deploy an AI Agent to Production

Quick answer
An agent in production is a long-running, expensive, non-deterministic job. Run it asynchronously, not inside a request.
Declare its scope: tools, credentials, data, and destinations. Anything not declared is a bug in the design.
Make every step idempotent and every run resumable, because retries and interruptions are normal.
Ship a kill switch and a spend limit before the first real task, not after the first incident.
Log every tool call with the run identity so an incident can be scoped from the trail.

What architecture does a production agent need?

A queue for work, a worker that runs the agent loop, a state store for run progress, a tool layer that enforces the allowlist, and an audit sink. The model API is one dependency, not the architecture.

Keep the loop out of the request path. A user-facing request that waits for an agent will time out, and a retried request will start a second run that duplicates the work. Return a run identifier and let the client poll.

POST /runs        -> { run_id, status: queued }
GET  /runs/{id}   -> { status, steps[], output, cost, errors }
POST /runs/{id}/cancel
worker: claim -> step -> checkpoint -> repeat -> finalise
every step: idempotency key + audit entry + budget check

What must be declared before launch?

Tools and their side effects, credentials and their scopes, data the agent may read, destinations it may send to, and the approvals required for consequential actions. This is the same inventory as a security review, expressed as configuration the runtime enforces.

If a capability is not declared, the runtime should deny it. A design that relies on the model not trying something is not a production design.

How do you make runs safe to retry?

Every step gets an idempotency key and a checkpoint. Before a step, write that it is starting; after, write the result. On retry, skip completed steps and re-run only the unfinished one. That turns a crash into a resume instead of a duplicate.

Tool calls that create external side effects need their own idempotency: a ticket creation should check for an existing ticket from the same run rather than creating another. Retries are where duplicates enter production data.

What does the first week look like?

Run a small set of real tasks with a human reviewing every output, and measure cost, latency, and failure modes. Then widen the task set while keeping the same review. Autonomy is earned per task class, not granted at launch.

Watch for the failure that looks like success: a run that completed and produced a plausible output that is wrong. That is why the review of outcomes, not just errors, is part of the rollout.

Where this bites vibecoders
The usual first deployment is a webhook that runs an agent with a personal token and no queue. It works in the demo, duplicates work on retry, and has no way to stop. Add the queue, the run identifier, and the kill switch before the second user.

Where AI coding assistants get this wrong

Running the agent inside a synchronous HTTP request.
Retrying a run without idempotency, creating duplicate side effects.
Granting the agent the deployer’s credentials.
Shipping without a cancel endpoint or spend limit.
Measuring errors but not wrong-but-successful outputs.
Expanding autonomy faster than review capacity.

Checklist

Queue work and return a run identifier.
Persist checkpoints and idempotency keys.
Declare and enforce tools, credentials, and destinations.
Add cancel and spend limits before launch.
Log every tool call to an external audit sink.
Review outcomes, not only errors, in the first week.

FAQ

Does the agent need its own database?

It needs durable run state: status, steps, checkpoints, and output. That can be a table in an existing database or a workflow engine. What it cannot be is the model’s context, because context does not survive a restart.

How do I handle long-running tasks?

Bound them. Split the task into steps that each fit a time budget, checkpoint between steps, and let a supervisor restart the run. An agent that runs for hours without checkpoints is one crash away from losing all the work.

What should the kill switch do?

Stop new runs, cancel in-flight runs at the next checkpoint, and revoke the agent’s credentials. A switch that only stops new work leaves the current run holding a live credential, which is the part you most want to stop.

How much autonomy is reasonable?

As much as you can review. A run that acts without approval needs an automated check that catches its errors, or a human who reviews the output before it takes effect. Autonomy without one of those is an unmonitored change process.