Emit gen_ai.client.token.usage and operation.duration metrics from the OTel middleware

[PratikDhanave]

What

The OTel middleware previously recorded token counts purely as span attributes (gen_ai.usage.*) via setUsage, and never measured run duration at all. It imported only otel/{attribute,codes,trace} and held a lone tracer — no Meter, no histograms.

Span attributes describe a single run and are not aggregatable across runs, so a spend/latency dashboard summing across many invocations saw nothing from the Go port. This change adds a Meter alongside the tracer and emits the two histograms the OTel GenAI semantic conventions define:

• gen_ai.client.token.usage (unit {token}) — two data points per run, one tagged gen_ai.token.type=input (InputTokenCount) and one output (OutputTokenCount).
• gen_ai.client.operation.duration (unit s) — one sample per run, measured from before span.Start.

Every data point carries the run's identifying attributes (gen_ai.operation.name, gen_ai.provider.name, gen_ai.agent.name), plus error.type when the run faulted. Recording is factored into a recordMetrics helper (mirroring setUsage) shared by both the normal post-loop path and the early-return path. Histogram construction errors are tolerated: NewMiddleware still returns a working tracer-only middleware and recordMetrics no-ops on a nil histogram.

Why (cross-SDK parity)

This aligns the Go port with the OTel GenAI semantic conventions and the Python/.NET references, which both emit these two histograms. Cross-language dashboards can now aggregate Go runs alongside the rest. Span-attribute accounting stays exactly as-is; metrics complement it (single-run detail vs. cross-run aggregation) rather than replacing it. Not a duplicate of the span PRs (#655/#646) or usage-accounting PRs (#594/#609/#551-554).

Tests

provider/otelprovider/otel_test.go gains two black-box tests driving an in-memory sdkmetric.ManualReader:

• TestOtel_Run_RecordsUsageAndDurationMetrics — runs the middleware over a fake RunFunc yielding UsageContent (100 input / 50 output), collects ResourceMetrics, and asserts the two gen_ai.client.token.usage data points (input=100, output=50 by gen_ai.token.type) with the shared attributes, plus one gen_ai.client.operation.duration sample.
• TestOtel_Run_MetricsIncludeErrorType — asserts error.type is present on the duration metric when the run faults.

Both fail before the change and pass after. go build ./..., go vet ./provider/otelprovider/..., and go test ./provider/otelprovider/... are green.

Open design questions

• Scope: should these live in the existing tracing middleware (as here) or a separate metrics middleware/config toggle? Kept together since they share the accumulated usage and the run boundary.
• API shape: MiddlewareConfig is unchanged — the Meter reuses SourceName for its instrumentation scope, matching the tracer. Open to a separate meter name if preferred.
• Follow-ups: AdditionalCounts and cache/reasoning sub-counters are not yet emitted as metric data points (the conventions only define input/output token-type dimensions); could be added if there's appetite.

[github-actions]

Cross-SDK Parity Review

PR in scope — adds gen_ai.client.token.usage and gen_ai.client.operation.duration histograms to the Go OTel middleware.

Upstream reference checked: python/packages/core/agent_framework/observability.py in microsoft/agent-framework (Python emits both histograms; no .NET equivalent was found in dotnet/ of the upstream repo).

Findings

The Go change aligns well with Python in metric names, gen_ai.token.type tag values (input/output), and error-type conditional attribution. However, two parity gaps were identified:

1. Missing explicit histogram bucket boundaries (meaningful divergence)

The Python SDK defines advisory bucket boundaries for both instruments:

# python/packages/core/agent_framework/observability.py
TOKEN_USAGE_BUCKET_BOUNDARIES = (1, 4, 16, 64, 256, 1024, 4096, 16384, 65536, 262144, ...)
OPERATION_DURATION_BUCKET_BOUNDARIES = (0.01, 0.02, 0.04, 0.08, 0.16, 0.32, ...)

def _get_token_usage_histogram():
    return get_meter().create_histogram(
        name=OtelAttr.LLM_TOKEN_USAGE,
        explicit_bucket_boundaries_advisory=TOKEN_USAGE_BUCKET_BOUNDARIES,
    )

The Go PR omits WithExplicitBucketBoundaries (or the advice.ExplicitBucketBoundaries option), leaving bucket shape to SDK defaults. Cross-language dashboards that aggregate Go alongside Python will see different bucket distributions for the same token/duration ranges, which partially defeats the goal stated in the PR description. Aligning bucket boundaries would complete the parity story.

2. Token usage unit string divergence (minor / spec alignment note)

Python uses unit: "tokens", while the Go PR uses unit: "{token}". The OTel GenAI semantic conventions spec defines the unit as {token} (UCUM custom-unit notation), so Go is actually more spec-correct here. This is a note for the Python team rather than a blocker — but it is a visible difference in metric metadata.

Summary

The core behavior (metric names, token-type tagging, error attribution, nil-histogram safety) is well-aligned with the Python upstream. The bucket-boundary gap is the most impactful change needed to achieve full cross-language dashboard compatibility.

Parity reviewer: GitHub Copilot CLI

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by Go API Consistency Review Agent · 34.1 AIC · ⌖ 5.72 AIC · ⊞ 5.7K · ◷