I could see how long the model call took in an OpenTelemetry span. My metrics export showed a counter I’d added during the same run. I looked for the inference duration histogram and came up empty. The counter told me the metrics exporter worked.
I filed AI SDK issue #24229 with the reproduction. The histogram shipped in @ai-sdk/otel 1.0.160 through PR #24354. That gave me a way to check what gen_ai.client.inference.duration actually measures.
I ran 25 calls on each version, five each for three generation delays, a stream and an error. A local mock model supplied the delays with timers, and I flushed the in-memory export after every call. The older version recorded zero histogram observations. The new version recorded 25, including all five errors.

A span attribute does not create a metric
I started with ai 7.0.157 and @ai-sdk/otel 1.0.157. The reproduction printed this:
metric names: ["repro.control"] duration attribute present: true An empty export could have sent me back to the exporter setup. Seeing repro.control gave me a reason to look at the integration first.
The duration I saw was gen_ai.client.operation.duration, an attribute on the span. To query the distribution I needed the integration to emit a histogram.
Follow both paths
The span and the histogram take separate paths. A TracerProvider sends spans to a span exporter. The metric needs a MeterProvider, a reader and a metric exporter.

In my reproduction I registered new OpenTelemetry() before setting the global meter provider. That order works with the shipped code because it looks up the meter when it records the duration.
Keep the experiment local
I pinned ai and @ai-sdk/otel at 7.0.157 and 1.0.157 for the first run, then 7.0.160 and 1.0.160 for the second configuration. Both used OpenTelemetry API 1.9.1, metrics SDK 2.8.0, trace SDK 2.7.1 and Node 24.22.2.
Both configurations ran on the same Linux server. Within each one I called MockLanguageModelV4 sequentially and flushed after each attempt. I disabled retries and left CPU and network settings alone.
| Group | Work inside the mock model | Repetitions |
|---|---|---|
| Short generation | Wait 20 ms, return text | 5 |
| Medium generation | Wait 80 ms, return text | 5 |
| Longer generation | Wait 320 ms, return text | 5 |
| Streaming | Wait 40 ms, emit text, wait 160 ms, finish | 5 |
| Failure | Wait 80 ms, throw an error | 5 |
An 80 ms timer can run a little late. I wanted to see the groups separate in the export, so I used delays far enough apart to see that.
Give the exporter a control measurement
The in-memory exporter let me read the measurements directly. Here’s how I set it up after registering tracing and the integration.
import { metrics } from '@opentelemetry/api'; import { AggregationTemporality, InMemoryMetricExporter, MeterProvider, PeriodicExportingMetricReader, } from '@opentelemetry/sdk-metrics'; const exporter = new InMemoryMetricExporter( AggregationTemporality.CUMULATIVE, ); const provider = new MeterProvider({ readers: [new PeriodicExportingMetricReader({ exporter, exportIntervalMillis: 60_000, })], }); metrics.setGlobalMeterProvider(provider); const control = metrics.getMeter('control') .createCounter('repro.control'); I incremented repro.control and flushed after every attempt. The issue reproduction has the tracing setup and mock model. To repeat the comparison, pin both AI SDK packages together.
Both counters ended at 25. Version 1.0.157 exported no inference histogram. Version 1.0.160 exported a histogram with 25 observations.
Check the recorded durations
After each call, I subtracted the previous cumulative sum for the same attribute set, checking that its count had increased by one. That gave me a duration in seconds for that call. I converted the medians to milliseconds for the table.

| Group | Median recorded inference duration |
|---|---|
| 20 ms generation | 21.8 ms |
| 80 ms generation | 81.2 ms |
| 320 ms generation | 320.8 ms |
| 40 + 160 ms stream | 202.3 ms |
| 80 ms failure | 81.2 ms |
The three generation groups landed close to their timer delays. The stream and the errors contributed their own measurements.
Let the stream finish
I had the mock wait 40 ms, emit text, then wait another 160 ms before closing. The caller consumed all of textStream.
First text reached my caller at a median of 43.6 ms across five calls. The histogram recorded a median of 202.3 ms for the full invocation. Those clocks start in different places, so I’d track time to first text as its own measurement.

For future reproductions I’d save the GenAI semantic convention revision alongside the package versions.
Include the calls that fail
The failing mock waited 80 ms and threw Error('controlled failure'). All five attempts were recorded with retries disabled.
I want to see those failures when I’m checking latency. The PR and its tests also cover cancellation and failed stream attempts during retries. My test exercised a thrown error.
The count is model attempts. One user action can cause several model calls, and retries can add more attempts. Local tool execution time is outside this duration.
Read the attributes beside the value
The error point had the operation and requested model alongside error.type. The provider was normalized from openai.chat to openai. The point can also carry a response model when the integration has one.
I chose the model IDs as labels for the mock groups. The exported point had no prompt or response text.
The updated integration docs show that telemetry.isEnabled: false turns off metrics for the call. The payload controls recordInputs and recordOutputs leave duration recording on. This histogram comes from the new integration.
Read cumulative counts once
I flushed after each attempt and chose cumulative aggregation. By the fifth flush, the snapshot for that group already included the previous four calls.

The export puts bucket counts into discrete intervals. I’d check how my backend represents them before writing a percentile query. The integration suggests boundaries in seconds, and the metrics SDK can override them.
I’d start the same way on a real service. Check the control counter and one successful call, then read the metric name, unit, attributes, count and sum in the export. Try one failed call and let one stream close. I would have missed that last 160 ms if I’d stopped reading at the first chunk.



