How to Monitor an AI Agent in Production

Quick answer
Errors are the least interesting signal. An agent can complete successfully and produce a wrong result, which looks like a healthy run.
Track three layers: run outcomes, tool calls, and cost and latency. Each answers a different question during an incident.
Alert on intervention rate and cost per run before you alert on individual errors.
Trace one run end to end with a run identifier, so logs, tool calls, and model calls join.
Review a sample of successful runs, not only failures, because that is where silent quality loss shows up.

What should you measure per run?

Status, duration, number of steps, tool calls by name, tokens by kind, cost, and whether a human intervened. Those seven numbers separate “slow”, “expensive”, “stuck”, and “wrong” without reading a single message.

Add an outcome signal where it exists: tests passed, deployment succeeded, review approved, ticket closed. Without an outcome, a completion metric only says the loop ended.

agent.runs.total{status}
agent.run.duration_ms
agent.run.steps
agent.tool.calls{tool,status}
agent.tokens{tokentype}
agent.cost.usd
agent.intervention.total
agent.outcome{tests,deploy,review}

Which alerts are worth waking someone for?

A rising intervention rate, cost per run above its baseline, a new tool being called, a destination that is not on the allowlist, and a run stuck beyond its time budget. These indicate a change in behaviour rather than a single failure.

Individual tool errors are usually noise: retries handle them. Alert on the pattern — the same tool failing repeatedly, or a retry rate that doubles — because that is what precedes a visible outage.

How do you trace a single run?

Use one run identifier as a trace or correlation identifier across the queue message, tool calls, model requests, and logs. Then a trace view shows the sequence: which step, which tool, which arguments, how long, and what came back.

Redact arguments at the source. A trace that contains customer data is a liability, and the redaction rule has to be applied where the span is created, not in a downstream processor that may be skipped.

What does quality monitoring look like?

Sample successful runs and score the outcome against the task. For code agents, that means whether the change was accepted, reverted, or fixed later. For support agents, whether the answer was correct and whether a human had to correct it.

The sample is the only way to catch degradation after a model or prompt change. Aggregate signals stay healthy while the outputs get subtly worse, and the first complaint arrives a week later.

Where this bites vibecoders
The usual monitoring is “did the job error out”, which reports 99 percent success while every third output is quietly wrong. Add an outcome signal and a weekly sample; the errors were never the interesting number.

Where AI coding assistants get this wrong

Alerting on individual tool errors and ignoring intervention rate.
Measuring completion without an outcome.
Logging prompts with customer data unredacted.
Lacking a run identifier that joins logs and traces.
Never reviewing successful runs for quality.
Changing the model or prompt without a before-and-after sample.

Checklist

Record status, steps, tools, tokens, cost, and interventions per run.
Add an outcome signal where one exists.
Alert on behaviour change, not single errors.
Correlate every span with a run identifier.
Redact arguments at span creation.
Sample successful runs weekly.

FAQ

Is this different from LLM observability?

LLM observability covers prompts, tokens, latency, and model quality. Agent monitoring adds the control plane: steps, tools, permissions, interventions, and outcomes. They share a trace but answer different questions.

How do I measure quality without a labelled set?

Use a proxy that exists in the workflow: was the change merged, was the ticket reopened, did a human edit the draft. Those are imperfect but available, and they move when quality moves.

What is a reasonable intervention rate?

It depends on the task, but a rising rate is the signal. Set a baseline from your first two weeks and alert on a sustained increase. A rate of zero for a task that should need judgement usually means nobody is checking.

Should I log the full conversation?

Only with a retention and redaction policy, and only if you will use it. Transcripts are large, contain sensitive data, and are rarely read. Prefer spans with structured fields for routine monitoring and transcripts for sampled or flagged runs.