Agents increasingly open pull requests, fix broken continuous integration (CI), upgrade dependencies, and remediate known security vulnerabilities (CVEs) without a developer watching every step. According to Gartner, 40% of enterprise applications will feature task-specific AI agents by 2026, up from less than 5% in 2025.

But platform, security, and engineering-operations teams often have no shared view of what those agents did. When an agent opens a pull request (PR), the reviewer may see the diff without the preceding instructions, tool calls, execution results, tests, or costs that produced it. Gartner also predicts more than 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear value, and inadequate risk controls. Better observability does not solve those problems on its own, but it gives teams the evidence needed to understand cost, evaluate outcomes, and enforce controls.

The principle underneath is simple: you can't govern what you can't see. This guide covers agent observability, the four pillars worth tracing, the OpenTelemetry standard that carries them, how traces support audit and cost controls, and how OpenHands brings those signals together around an agent run.

What is AI agent observability?

AI agent observability is the collection of traces, logs, metrics, outcomes, and attribution data that helps teams understand and govern what happened during an agent run. The quality of that view depends on how completely the agent, tools, runtime, and surrounding workflow are instrumented. It extends the traces, metrics, and logs platform teams already collect for services, adding the agent-specific signals such as the instructions it received, the observable steps it took, the tools it called, the resulting changes, validation outcomes, and cost.

Three traits separate it from a raw application log:

• It is run-scoped: The unit is a full agent run from goal to result, not a single request or model call, so it reads as a coherent timeline.

• It is behavioral, not just operational: It records the agent’s observable actions and the evidence used to evaluate the result, not only whether the process stayed available or returned a successful status code.

• It is attributable: Each run ties back to the workflow that started it, the user or system that triggered it, the models it called, and the resources it touched.

Observability is foundational to effective control: Teams cannot reliably scope permissions, investigate failures, attribute cost, or define appropriate review gates when agent activity is poorly instrumented. A reviewer does not need access to a model’s private reasoning, but they do need visibility into the instructions the agent received, the tools and systems it used, the validation checks it ran, the changes it produced, and the cost of the workflow.

Why agents are harder to observe than models and traditional services

Application performance monitoring (APM) and large language model (LLM) observability are often centered on requests, model calls, services, and distributed dependencies. Agent workflows add another layer: a run may loop through planning, model calls, tools, environment changes, and validation over tens or hundreds of steps. Important failures can therefore emerge across the workflow rather than inside any one model call.

A model call can look healthy while the run it belongs to quietly does the wrong thing: a confident plan, then a silent shell failure, then a run that reports success. Some agent failures can present as successful runs when the model reports completion despite failed tools, incomplete changes, or unmet validation criteria.

Several properties make agent runs hard to pin down:

• Non-determinism: The same goal and inputs can produce different paths, so one trace doesn't represent all runs.

• Long tool chains: A run can span hundreds of tool calls across the file system, shell, and network, any of which can fail.

• Parallel agent fleets: Once agents run on schedules and events, many are in flight at once, with no single terminal to watch.

• Scattered cost: Spend spreads across models, providers, and runs, so no line item shows what a workflow costs.

• Real side effects: Agents write files, push branches, call application programming interfaces (APIs), and touch production systems, so an unobserved run isn't harmless.

None of this shows up in request-shaped monitoring, so the practical question is what to capture across a full run.

What to trace across an agent run: the four pillars

Treat a run as an ordered timeline. Four things worth capturing answer what a platform or security team asks about any agent: what was it asked to do, what did it do, did it work, and what did it cost.

These four pillars are a practical baseline rather than an exhaustive schema. Security-sensitive deployments may also need identity, permission decisions, network activity, policy evaluations, artifacts, and approval events.

Plans, prompts, and execution traces

The first pillar is the observable sequence of the run: the goal, instructions, generated plan where one is exposed, transitions between steps, model calls, and parent-child spans. This lets a reviewer follow how the workflow progressed from request to result. It does not guarantee access to a model’s private internal reasoning, and generated explanations should not be treated as a perfect account of why the model acted.

Tool calls and actions

The second pillar is what the agent did to the world: its tool invocations with inputs and outputs, the shell commands it ran, the files it edited, the APIs it hit, and the pull requests it opened. This is especially important for security and platform teams because it provides a concrete record of which repositories, credentials, tools, and systems the workflow touched.

Outcomes and evaluation

The third pillar is whether the work met its acceptance criteria. Outcomes can include deterministic checks (did the tests pass, the build succeed, the linter stay green) alongside model-assisted evaluation (an LLM judging whether the change matches the request) where appropriate. This catches failures that look like success: a run that reports "done" while its tests are red, or a fix that quietly breaks an adjacent path. Model-based judgments can add useful signals, but they should not substitute for deterministic verification on checks that can be expressed directly.

Cost, latency, and attribution

The fourth pillar is resource usage: token consumption, model and tool latency, runtime duration, retries, and cost attributed to the run, workflow, model, and owning team. This makes it possible to identify expensive workflows, compare model choices, and detect unusual usage patterns earlier.

Read together on one timeline, the four pillars let a human reconstruct what an agent did, whether it worked, and what it cost, in one view instead of four separate tools.

Trace once, read anywhere: the OpenTelemetry standard for agents

Capturing those four pillars is only useful if the data is portable, which is what OpenTelemetry solves. It provides a vendor-neutral telemetry model and ecosystem of exporters, making it easier to send traces, metrics, and logs to different compatible backends without tying instrumentation to a single observability vendor.

For agents, the OpenTelemetry project maintains a set of GenAI semantic conventions:  a common vocabulary for describing model calls, tool executions, and agent operations as spans. They define well-known operation names such as chat, execute_tool, invoke_agent, and create_agent, plus attributes for tool names, agent identities, and token usage (gen_ai.usage.input_tokens and gen_ai.usage.output_tokens). The goal is to give agent frameworks and observability backends a more consistent vocabulary for common operations, making cross-tool telemetry easier to normalize over time.

Three caveats matter. First, as of mid-2026, the GenAI semantic conventions remain under active development, so operation names, attributes, and documentation may continue to change. Second, different libraries implement different subsets of the conventions and different versions of them, so two frameworks tracing the same kind of call don't always populate the same attributes yet. Third, OpenTelemetry is a general industry standard, not any one company's product: it defines how to describe a trace, not what your agents do or how they're governed.

Where agent traces feed audit, policy, and cost control

Traces provide evidence. Governance systems use that evidence alongside identity, policy engines, approvals, and budget controls.

• Audit trails: A queryable history of runs, where actions are logged and tied to the workflow and user who triggered them, lets a security or compliance team answer "who ran what, against which systems, and when."

• Least-privilege access policy: Visibility into repositories, tools, and secrets helps teams design and validate least-privilege policies. Enforcing those boundaries still requires scoped identities, credentials, authorization rules, and isolated execution. Agents should run as their own scoped principals, not under a developer's full credentials.

• Human review gates: Outcome and risk data can help teams identify narrowly defined workflows that may qualify for more streamlined review, while higher-impact changes remain behind mandatory human approval. Name those checkpoints explicitly instead of assuming a run is fully autonomous or fully manual.

• Budgets and cost attribution: Per-run cost data can roll up into team or workflow-level reporting, alerts, limits, and approval policies. The exact enforcement mechanism depends on the platform, but attribution is what makes agent spend governable, rather than an aggregate provider bill.

These are general frameworks: industry bodies publish zero-trust and least-privilege guidance for autonomous agents, not any single product's feature list. All four depend on the same thing underneath: an attributable trace of each run.

Best practices for AI agent observability

A practical rule for keeping agents governable is to instrument early and avoid scaling workflows faster than your ability to observe and investigate them.

• Instrument from the first run: Turn on tracing before you have a fleet, not after an incident, so you never reconstruct behavior you didn't record.

• Prefer portable telemetry standards: Where supported, emit traces in OpenTelemetry-compatible formats so agent and service data can be correlated and backends can change without rebuilding every integration.

• Trace outcomes, not just logs: Capture validation outcomes, not just execution logs, so a reported “done” can be checked against objective evidence.

• Attribute cost per agent and team: Tie token spend and run duration back to the workflow and owner, so cost is a budgetable line item, not an aggregate mystery.

• Keep clear ownership for autonomous workflows: Every unattended workflow should map to an accountable team or owner, with explicit review gates for changes that require human approval.

• Validate on one workflow before scaling: Prove observability and controls on one repeatable workflow, then expand, rather than turning on a hundred agents and hoping the traces hold up.

Do these consistently and observability becomes the control surface you run agents from. That's where a specific platform makes the difference.

How OpenHands supports observable, governed agent workflows

Many coding-agent workflows begin with individual developers, but the visibility requirements change when those agents move into shared, scheduled, or event-driven use. OpenHands treats the run itself as a first-class part of the platform: you can't govern what you can't see.

OpenHands supports that progression across Agent Canvas, Cloud, and Enterprise. Agent Canvas is the local-first workspace where developers can run agents against real code and review the conversation, supported tool activity, execution output, and file changes produced during the workflow. OpenHands Cloud provides hosted execution for supported workflows, including scheduled or event-driven runs where available.

OpenHands Enterprise adds organization-level capabilities such as role-based access controls, centralized authentication, audit logs and workflow visibility, usage and cost controls, and self-hosted deployment options.

This matters most in the outer loop, where agents may run without a developer watching every step and telemetry becomes an important part of reconstructing what happened. Underneath, the OpenHands SDK can emit OpenTelemetry-compatible traces for supported agent, model, and tool activity when observability is configured, allowing teams to correlate agent telemetry with the rest of their observability stack. Run a workflow like an automated vulnerability fixer across many repositories, and that visibility makes it easier to investigate runs, review changes, attribute usage, and support audit requirements than relying only on the final pull request.

Making observability the starting point for enterprise agent adoption

Scaling autonomous agents responsibly is not only about running more of them. It is about being able to understand what those workflows did, evaluate their outcomes, and apply controls as usage expands. Observability isn't a dashboard you bolt on after deployment. It is part of the operating model for governed agent workflows.

So start there. Pick one outer-loop workflow, such as a scheduled dependency upgrade or vulnerability-remediation run. Put its trigger, instructions, observable steps, tool calls, validation outcomes, and cost into a shared view before scaling. Make sure your team can reconstruct the material events of one unattended run: what started it, which tools and systems it touched, what changed, whether validation passed, and what it cost. Then expand with the same visibility and controls in place.

OpenHands Enterprise provides the governance and visibility layer for teams ready to standardize those controls across agent workflows.

Frequently asked questions about AI agent observability

What is the difference between AI agent observability and LLM observability?

LLM observability measures individual model calls: tokens, latency, and error rates on one request. AI agent observability measures a full run, the multi-step workflow of model calls, tool use, execution, validation, and side effects. A run-level view provides the context needed to evaluate whether the agent completed the intended work and where failures occurred.

What should you monitor when running AI agents in production?

Monitor the run’s trigger and owner, instructions and observable step sequence, tool calls and side effects, validation outcomes, cost and latency, permissions used, and any human approval events. The exact schema depends on the risk of the workflow.

How do you control and attribute the cost of AI agents at scale?

Capture token spend and run duration per run, then attribute them to the model, workflow, and owning team, so cost is a budgetable line item, not an aggregate bill. From there, teams can apply alerts, limits, or approval policies depending on what their platform supports. On OpenHands, cost attribution and budgets are part of the Enterprise Agent Control Plane; compare plans and tiers on the pricing page.

How do you secure and govern autonomous AI agents?

Give each agent workflow a scoped workload identity or credential set with access only to the repositories, tools, and secrets required for the task. Pair those access boundaries with durable logs, explicit approval gates, and enough attribution to determine which user, system, or workflow initiated the activity. OpenHands Enterprise is designed to associate supported workflow and agent activity with the initiating user or system, subject to the configured identity, logging, and integration setup.

What is OpenTelemetry's role in AI agent observability?

OpenTelemetry is the vendor-neutral, open-source standard for traces, metrics, and logs, and its GenAI semantic conventions add a common vocabulary for model calls, tool executions, and agent operations as spans. Instrumenting with it sends agent telemetry to any compatible backend instead of one vendor. The conventions are still evolving, so teams should expect schema changes and uneven support across frameworks. The OpenHands SDK supports OpenTelemetry-compatible tracing for supported activity when observability is configured.

About OpenHands

OpenHands is the open-source platform for building and running AI coding agents, with the interface, automations, and control layer needed to go from a single local agent to a system running across an entire organization. The mission is to make agent-based software development accessible, transparent, and controllable by default. That starts in the open. The core framework is open source, giving developers and platform teams full visibility into how agents execute work and interact with their systems. The project has over 80,000 GitHub stars, over 9 million downloads, and contributions from hundreds of developers. OpenHands is used by engineers at large enterprises and fast-growing startups to build, run, and scale AI coding agents across real software engineering workflows. The long-term vision is to become the full stack AI coding agent platform for software engineering. Not just helping developers write code, but running meaningful parts of the software lifecycle.