Most advice on how to monitor ai agents in production starts with picking a platform: compare dashboards, pick one, connect it. That order is backwards.
A dashboard shows you whatever you told it to watch. If nobody has written down what the agent is supposed to do, what it may touch and who answers when it goes wrong, the dashboard is a well-lit view of nothing in particular.
Monitoring an agent is an operating routine with four parts: map the workflow, trace every run, alert on a small set of signals, and keep a named human reviewing. Here is how each part goes.
What does monitoring an AI agent in production actually involve?
You capture every run as a trace, track cost, tool and quality signals for each one, alert when those signals cross thresholds, and have a person review samples of real runs on a schedule. Tooling collects the data. A named owner decides what it means and what changes.
That last part is what gets skipped. Tools are easy to buy and easy to leave unattended. An agent with no owner drifts quietly, and the first person to notice is usually a customer.
Why does uptime and latency monitoring miss agent failures?
Because an agent can finish a run, return a clean success code, and still have done the wrong thing. It may have looked up the wrong account, skipped a required step or sent a confident answer built on a bad retrieval. Nothing errored, so nothing alerted.
The behavior is also non-deterministic. The same input can take a different path on Tuesday than it did on Monday. Agent monitoring has to follow session-level behavior across branching workflows, often 10 to 50 or more decision points per task, and judge end-to-end completion rather than whether an API call returned 200.
Step 1: Map the workflow before you instrument it
You cannot alert on behavior nobody defined. Before any tracing, write a one-page description of the job.
Write down what correct looks like before any tool is asked to watch for it.
- What done correctly means. Describe the outcome in terms an ops lead could check by hand: the right record updated, the right reply sent, the right queue chosen.
- Which tools it may call. List every system the agent can read from or write to, and mark which actions are reversible.
- Where it hands off. Name the conditions that send a run to a person: low confidence, refund above a limit, an angry customer, a missing record.
- Who owns it. One named person, not a team alias.
This page becomes your test for every review later. When a reviewer asks whether a run was wrong, the answer is on the page.
Step 2: Capture traces and which metrics should you alert on?
A trace is a complete agent task from start to finish, such as handling one user request. Spans are the individual steps inside it, like a model call or a data retrieval. The Microsoft lesson on agent observability and evaluation uses the same definitions.
Emit traces with OpenTelemetry where you can. The same telemetry can then go to more than one backend, so you are not tied to a vendor before you know what you need.
Signals worth alerting on
Start small. Six signals cover most of what goes wrong:
- Tokens and cost per run. A sudden rise usually means a loop or a bloated prompt.
- Tool call failure rate. One suggested alert is tool failures divided by tool calls above 0.05 for any single tool.
- Loop iterations. Alert when the average exceeds twice your baseline, which means you need a baseline first.
- Context window use. One suggested alert is p95 utilization above 0.8.
- Latency percentiles. Watch p95, not the average.
- End-to-end completion. The share of runs that reach the outcome you defined in Step 1.
Treat those thresholds as starting points and tune them against your own traffic.
Do not log full prompts and responses in production. They contain user data. Log lengths and token counts, and keep content only in a controlled review sample.
Step 3: Where should a human review agent behavior?
Review has two jobs. Monitoring covers guardrails, prioritizing issues and root cause analysis. Improving covers remediation, knowledge base fixes, evaluations and fine-tuning. Name who does which, even if it is the same person at first.
A named person reading real runs each week catches what no metric will.
Set a routine. Each week, the owner reads a sample of completed runs against the Step 1 page, plus every run that escalated or tripped an alert. Early on, sample heavily. Reduce only once the failures you find become rare and minor.
Scale oversight with risk. An agent that tags inbound tickets can be sampled lightly. One that issues refunds or edits customer records needs approval before the action, not review after it.
Every failure you find should end in a change: a prompt edit, a knowledge base fix, a tool restriction or a new test case. A review that produces notes and no changes is a ritual.
Step 4: How do you handle errors, drift and documentation over time?
Build error handling on day one, not after the first incident. Every tool call needs a defined failure path: retry, fall back, or hand to a person. Every agent needs a kill switch that one named person can use without a deploy.
Drift arrives from two directions. The model provider updates a model, or your data changes: new products, new policies, new ticket types. Compare current completion rates and review findings against your baseline, and rerun a fixed set of known cases after any model or prompt change.
Document all of it: the workflow page, alert thresholds, escalation rules, the kill switch and the review log. Your company should own that documentation, so the system survives a change of vendor or staff. It is how we approach AI agent development, and you can see how projects are scoped in our client case studies.
FAQ
How do you monitor an LLM in production?
Trace each call with its token counts, latency and cost, and log lengths rather than raw content. Alert on spikes in cost or failures, then have a person read a sample of outputs regularly to catch quality problems that metrics cannot see.
How do you measure AI agent performance?
Measure whether runs reach the outcome you defined, not whether the API responded. Track end-to-end completion, tool call failure rate, loop iterations against baseline and cost per run, then check those numbers against human review of sampled runs.
Why can't traditional monitoring track AI agents?
Traditional monitoring checks whether a service is up and fast. An agent can be up, fast and wrong, because its path varies run to run and its failures are semantic. Catching them takes traces of each decision and human judgment on outcomes.
Where to start
Monitoring is a routine with an owner, not a product you install. The tools matter less than the page that defines correct behavior and the person who reads the runs.
This week, pick one agent workflow, write its one-page definition and name its owner. If you want a read on how ready your operations are for this, take the free AI Ops Score.