AI Agent Tracing

This demo opens up the black box of an AI agent. A multi-step agent answers a user request, and every step is captured as an OpenTelemetry trace — so you can see exactly what happened, how long it took, and what it cost.

What it demonstrates

• End-to-end traces — one user request becomes a tree of spans: the agent loop, each model call, each tool call, and retrieval steps.
• GenAI attributes — model name, token counts, temperature, and finish reason recorded on every model span using OpenTelemetry’s generative-AI semantic conventions.
• Latency breakdown — instantly see whether time went to the model, a slow tool, or retrieval.
• Cost per request — token metrics roll up into a cost figure you can trend over time.

How it works

1. The agent loop is instrumented so each model call, tool call, and retrieval is wrapped in a span.
2. Telemetry flows through the OpenTelemetry Collector and into a backend (Grafana/Tempo in this demo).
3. A dashboard shows traces, token usage, latency percentiles, and error rates per agent and per tool.

Why it matters

“The agent was slow” or “the answer was wrong” stop being mysteries. With traces, you point at the exact span that caused the problem — and because it’s all open standards, you’re never locked into one observability vendor.

This is the foundation of our AI Observability package: OpenTelemetry, dashboards, alerts, and AI-specific metrics in three weeks.