To monitor agents built with Pydantic AI or Google ADK, export the OpenTelemetry traces both frameworks already emit to a monitoring backend, and attach a user and session ID to each run. For production monitoring of whole conversations, Agnost AI is the pick: it receives those spans over OTLP, groups them by user and session, judges every conversation against plain-language rules, and alerts in Slack or email when something breaks. Pydantic Logfire is the native tool for inspecting Pydantic AI runs, and Google Cloud Trace is the built-in destination for ADK.
Both frameworks follow the OpenTelemetry semantic conventions for generative AI, so the same backend can read either one. The difference is in setup: Pydantic AI instruments through Logfire's SDK (or plain OpenTelemetry), and ADK exports through its own OTLP setup plus a plugin if you want user IDs and transcripts on the root span.
Disclosure: We make Agnost AI. Facts about Pydantic AI, Google ADK, and other tools come from their own docs, checked on November 21, 2026.
Quick picks
- Production monitoring of whole conversations, with alerts and fix PRs: Agnost AI
- Inspecting Pydantic AI runs, retries, and validation during development: Pydantic Logfire
- ADK on Google Cloud, with traces next to your other GCP services: Google Cloud Trace
- An open-source tracing and eval workbench for ADK: Phoenix or Langfuse, both in ADK's integrations catalog
Quick comparison
| Tool | Pydantic AI | Google ADK | Best for |
|---|---|---|---|
| Agnost AI | Logfire SDK with send_to_logfire=False, OTLP to Agnost | ADK's OTLP export plus a small plugin for user and transcript | Judging whole production conversations, alerts, and fix pull requests |
| Pydantic Logfire | Native: logfire.instrument_pydantic_ai() | Not covered in the docs checked | Development debugging of Pydantic AI runs |
| Google Cloud Trace | Not covered in the docs checked | Built in: --otel_to_cloud | ADK teams on Google Cloud |
| Phoenix, Arize AX, Langfuse, W&B Weave, MLflow Tracing, Datadog | Any OTLP backend via OpenTelemetry | Listed in ADK's integrations catalog | Tracing and evals in a tool you already use |
| Any OTLP backend (Jaeger, Grafana Tempo, otel-tui) | Agent.instrument_all() or the Logfire SDK | OTEL_EXPORTER_OTLP_TRACES_ENDPOINT | Raw traces in your own stack |
How Pydantic AI observability works
Pydantic AI emits OpenTelemetry traces of every run, model request, and tool call. Its docs describe the instrumentation as plain OpenTelemetry that follows the GenAI semantic conventions. Logfire is the default way to turn it on: logfire.configure() and then logfire.instrument_pydantic_ai().
Because the spans are standard OpenTelemetry, you choose where they go:
- Logfire's hosted backend, the default.
- Another backend through the Logfire SDK: set
OTEL_EXPORTER_OTLP_ENDPOINTand calllogfire.configure(send_to_logfire=False). - No Logfire at all: set up the OpenTelemetry SDK with a
BatchSpanProcessorand an OTLP exporter, then callAgent.instrument_all().
Send Pydantic AI traces to Agnost AI. This follows Agnost's Pydantic AI guide:
pip install pydantic-ai logfire import os import logfire os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://otel.agnost.ai" os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = f"X-Agnost-Org-ID={os.environ['AGNOST_ORG_ID']}" logfire.configure(send_to_logfire=False) # don't dual-send to Logfire's backend logfire.instrument_pydantic_ai() # Logfire copies baggage onto every span, so user.id lands on each one. with logfire.set_baggage(**{"user.id": user_id}): result = agent.run_sync(prompt, conversation_id=conversation_id) Pass your own conversation_id on each run so every turn of a chat lands in the same conversation; otherwise Pydantic AI generates a new one per run. Don't set LOGFIRE_TOKEN, which turns the hosted backend back on. For validation retries, typed outputs, and "valid but wrong" results, see the full Pydantic AI monitoring guide.
How Google ADK observability works
ADK has built-in logging, metrics, and traces. Traces follow the OpenTelemetry GenAI conventions and export in OTLP format. An agent run is the root span, with child spans for model and tool calls:
-
invoke_agent {agent.name}: one agent interaction. -
invoke_workflow {workflow.name}: a multi-step workflow. -
generate_content {model.name}: a model call, including token usage. -
execute_tool {tool.name}: a tool or function call.
To send them to Google Cloud Trace, run adk web --otel_to_cloud (or adk api_server), or use the Google Cloud exporter in code. To send them anywhere else, set OTEL_EXPORTER_OTLP_TRACES_ENDPOINT. ADK's docs warn that the general OTEL_EXPORTER_OTLP_ENDPOINT also sends metrics and logs.
Send ADK traces to Agnost AI. This follows Agnost's Google ADK guide. Install the HTTP exporter, which ADK's built-in OTLP setup needs:
pip install google-adk opentelemetry-exporter-otlp-proto-http export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="https://otel.agnost.ai/v1/traces" export OTEL_EXPORTER_OTLP_TRACES_HEADERS="X-Agnost-Org-ID=your-org-id" export OTEL_SERVICE_NAME="my-adk-agent" export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT="SPAN_ONLY" On ADK 2.x, prompts, answers, and tool arguments stay off spans unless OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=SPAN_ONLY is set. adk web and adk api_server read these variables directly; in your own process, call maybe_set_otel_providers() once at startup.
ADK writes the session ID onto its spans as gen_ai.conversation.id, so each ADK session becomes one Agnost conversation with no extra code. ADK records the user ID only on log records, and its root invoke_workflow span carries no message text, so add a short plugin that copies both onto the root span:
import json from google.adk.plugins.base_plugin import BasePlugin from opentelemetry import trace def _text(content): return "".join(p.text for p in (content.parts or []) if p.text) if content else "" class AgnostPlugin(BasePlugin): """Adds user.id and each turn's input and answer to ADK's root span.""" def __init__(self): super().__init__(name="agnost") async def on_user_message_callback(self, *, invocation_context, user_message): span = trace.get_current_span() # the root invoke_workflow span span.set_attribute("user.id", invocation_context.session.user_id) span.set_attribute("gen_ai.input.messages", json.dumps( [{"role": "user", "parts": [{"type": "text", "content": _text(user_message)}]}])) async def on_event_callback(self, *, invocation_context, event): if event.is_final_response() and _text(event.content): trace.get_current_span().set_attribute("gen_ai.output.messages", json.dumps( [{"role": "assistant", "parts": [{"type": "text", "content": _text(event.content)}]}])) Register it with Runner(..., plugins=[AgnostPlugin()]), or App(plugins=[...]) when you serve an App with adk web. The plugin only sets span attributes and never changes how the agent runs. If your app already sets its own TracerProvider, ADK leaves it alone, so add an Agnost OTLPSpanExporter to that provider instead.
Agnost AI
What it is. Monitoring for AI agents in production: chat agents, voice agents (through transcripts), copilots, and MCP servers. It is OpenTelemetry-native and works with Pydantic AI, Google ADK, the OpenAI Agents SDK, LangChain and LangGraph, Mastra, the Vercel AI SDK, and Agno.
Best for. Teams that need to know which user conversations went wrong, not only which spans errored.
What it does with Pydantic AI and ADK traces. It groups runs by user and session and judges whole conversations, not single traces. It detects failures, user frustration, intents, tool-call errors, loops, and silent wrong answers. You write rules in plain language, and once a conversation has been idle 15 minutes an LLM marks each assistant message pass or fail, with no sampling. Alerts go to Slack or email, and Auto Improve opens a pull request with a fix tested against past runs for your team to review. Each conversation also opens to its spans and tool calls, with arguments and results when content capture is on.
Limits. Not open source. No cost tracking, offline datasets, experiments, or CI gating. Verdicts are pass or fail, not scores, and judging runs in batches, so keep real-time error alerts elsewhere.
Pricing. Free (1,000 events a month, 7-day retention), Starter $49 a month (10,000 events), Pro $499 a month (1,000,000 events), Enterprise custom with self-hosted VPC.
Pydantic Logfire
What it is. An AI engineering platform from the team that maintains Pydantic Validation and Pydantic AI, built on OpenTelemetry.
Best for. Pydantic AI developers who want runs, model requests, tool calls, and validation retries visible with two lines of code.
Limits. Pydantic's docs describe it as a commercial product with a free tier; self-hosting is on the enterprise tier. It shows execution; whether a conversation met the user's goal is up to you to define.
Google Cloud Trace
What it is. Google Cloud's tracing service, and ADK's built-in destination through --otel_to_cloud or the Google Cloud exporter.
Best for. ADK teams already deployed on Google Cloud who want agent spans next to the rest of their services.
Limits. It shows spans, latency, and errors per run. It does not group runs into user conversations or judge them.
ADK's other observability integrations
ADK's integrations catalog lists, among others, AgentOps, Arize AX, Phoenix, Langfuse, LangWatch, W&B Weave, MLflow Tracing, Datadog, Galileo, Future AGI, Freeplay, and a BigQuery Agent Analytics plugin. Pick one of these when you want tracing plus offline datasets and experiments in a tool your team already uses. Because Pydantic AI emits standard OpenTelemetry, any of them that accepts OTLP can read Pydantic AI spans too; check each vendor's docs for a dedicated integration.
How to choose
| If your team… | Start with |
|---|---|
| Needs to know which production conversations failed, with alerts and fix PRs | Agnost AI |
| Builds with Pydantic AI and wants the fastest view of a single run | Pydantic Logfire |
| Runs ADK on Google Cloud | Google Cloud Trace, plus Agnost AI for conversations |
| Wants open-source tracing with offline evals | Phoenix or Langfuse |
| Already runs Datadog | Datadog, plus Agnost AI for conversations |
Many teams keep the framework's native tool for development and add a conversation-level layer for production. The two read the same spans, so you don't have to choose one instrumentation.
What to monitor once traces arrive
- Conversation identity. Pass a stable
conversation_idto Pydantic AI and reuse the ADKsession_idacross turns, or every turn becomes its own conversation. - Tool outcomes. A tool returning text is not proof that a booking or refund happened. Record the system-of-record result.
- Validation and retries (Pydantic AI). A schema-valid output can still be wrong; count retries separately from user demand.
- Multi-agent routing (ADK). Check that sub-agents receive the context they need, and watch for loops between agents.
For more on what to watch, see conversation-level vs trace-level online evals and the agent observability guides by framework.
Related reading
- Pydantic AI monitoring and observability guide
- How to instrument an AI agent with OpenTelemetry
- Best online evaluation tools for LLM agents in production
- Agent observability guides by framework
Frequently asked questions
How do I monitor agents built with Pydantic AI or Google ADK?
Both frameworks emit OpenTelemetry traces that follow the GenAI semantic conventions, so you export them over OTLP to a monitoring backend. For Pydantic AI, call logfire.instrument_pydantic_ai() and point the OTLP exporter where you want the spans. For Google ADK, set OTEL_EXPORTER_OTLP_TRACES_ENDPOINT and call maybe_set_otel_providers(), or use --otel_to_cloud for Google Cloud Trace. For production monitoring of whole conversations, Agnost AI receives those spans, groups them by user and session, judges each conversation against plain-language rules, alerts in Slack or email, and can open a pull request with a fix tested against past runs.
What is the best observability tool for Pydantic AI?
Pydantic Logfire is the native option, built by the team behind Pydantic AI, and is the quickest way to inspect runs, model requests, tool calls, and validation retries. It is a commercial product with a free tier, and it can be self-hosted on its enterprise tier. Because Pydantic AI's instrumentation is plain OpenTelemetry, you can send the same spans to any OTLP backend instead; Agnost AI is the pick when you want production conversations judged and failures turned into alerts and pull requests.
How do I add observability to a Google ADK agent?
ADK has built-in logging, metrics, and OpenTelemetry traces with invoke_agent, invoke_workflow, generate_content, and execute_tool spans. Run adk web or adk api_server with --otel_to_cloud to send them to Google Cloud Trace, or set OTEL_EXPORTER_OTLP_TRACES_ENDPOINT and call maybe_set_otel_providers() to send them to any OTLP backend. ADK's integrations catalog also lists tools such as Arize AX, Phoenix, Langfuse, W&B Weave, AgentOps, MLflow Tracing, and Datadog.
Does Google ADK support OpenTelemetry?
Yes. ADK implements the OpenTelemetry semantic conventions for GenAI and exports in OTLP format. An agent run is the root span, with child spans for model calls and tool executions. Use OTEL_EXPORTER_OTLP_TRACES_ENDPOINT for traces only; the general OTEL_EXPORTER_OTLP_ENDPOINT also sends metrics and logs.
Can I use Pydantic AI without Logfire?
Yes. Pydantic AI's docs show two ways: keep the Logfire SDK but call logfire.configure(send_to_logfire=False) and set OTEL_EXPORTER_OTLP_ENDPOINT to another backend, or skip Logfire entirely, set up the OpenTelemetry SDK with an OTLP exporter yourself, and call Agent.instrument_all().
Is Google Cloud Trace enough to monitor an ADK agent in production?
It is enough to see each run's spans, latency, and errors. It does not group runs into user conversations or judge whether the user got what they asked for. If you need that, send the same ADK spans to a conversation-level tool such as Agnost AI, which flags failures, user frustration, tool-call errors, and loops across whole conversations.
Why do my ADK traces in Agnost have no prompts or answers?
On ADK 2.x, message content stays off spans unless OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT is set to SPAN_ONLY before ADK starts. Agnost also needs a small ADK plugin that writes the user ID and each turn's input and answer onto the root invoke_workflow span, because ADK records the user ID on log records only.
Originally published on agnost.ai.