Skip to content

Request include_usage for OpenAI chat streaming so streamed runs report token usage#609

Open
PratikDhanave wants to merge 1 commit into
microsoft:mainfrom
PratikDhanave:openai-stream-include-usage
Open

Request include_usage for OpenAI chat streaming so streamed runs report token usage#609
PratikDhanave wants to merge 1 commit into
microsoft:mainfrom
PratikDhanave:openai-stream-include-usage

Conversation

@PratikDhanave

Copy link
Copy Markdown
Contributor

What

In the OpenAI Chat Completions streaming path, set stream_options.include_usage=true before starting the stream, unless the caller already supplied a value.

Why

buildCompletionParams never set StreamOptions.IncludeUsage, and the SDK omits the field when unset (it is a param.Opt[bool] with omitzero). The OpenAI API only emits the final usage chunk for streamed completions when stream_options.include_usage=true. Without it, the streaming branch's chunk.JSON.Usage.Valid() guard never fires, so every streamed run reported zero token usage — while the non-streaming path reported usage normally.

This restores parity with the .NET and Python OpenAI chat clients, which enable usage reporting for streamed completions so the same token accounting is available regardless of streaming mode. The existing addUsage aggregation already consumes the resulting usage chunk unchanged.

The caller's explicit choice is respected: if StreamOptions.IncludeUsage is already set (including to false), the provider does not override it. The non-streaming path is untouched.

Tests

  • Extended the streaming tests in chat_test.go to assert the outbound request body now contains stream_options.include_usage=true, and that the aggregated UsageContent carries non-zero token counts.
  • Added TestChatStreamingIncludeUsage_RespectsCallerOverride, asserting that a caller-supplied include_usage=false is not overridden.

go build ./..., go vet ./provider/openaiprovider/..., and go test ./provider/openaiprovider/... all pass.

Copilot AI review requested due to automatic review settings July 23, 2026 05:36
@PratikDhanave
PratikDhanave requested a review from a team as a code owner July 23, 2026 05:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the OpenAI Chat Completions streaming request path to default stream_options.include_usage=true so streamed runs receive (and can aggregate) token usage from the final stream chunk, matching the non-streaming behavior and other SDKs.

Changes:

  • Default stream_options.include_usage to true for streaming requests when the caller did not set it.
  • Preserve caller intent by not overriding an explicitly provided include_usage value (including false).
  • Extend streaming tests to assert the outbound request includes stream_options.include_usage and to cover the caller-override case.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
provider/openaiprovider/chat.go Sets StreamOptions.IncludeUsage to true by default in the streaming branch when unset, enabling usage reporting from the final stream chunk.
provider/openaiprovider/chat_test.go Updates streaming request body expectations and adds a test ensuring a caller-supplied include_usage=false is respected.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

The OpenAI Chat Completions API omits the final usage chunk for streamed
runs unless stream_options.include_usage is set, so every streamed run
reported zero token usage while the non-streaming path reported usage
normally. Set stream_options.include_usage=true before starting the stream
unless the caller already supplied a value, matching the token accounting
the .NET and Python OpenAI chat clients get from streamed completions.
@PratikDhanave
PratikDhanave force-pushed the openai-stream-include-usage branch from fc3dc20 to fb4c2c8 Compare July 23, 2026 15:48
@github-actions github-actions Bot added the parity-approved Go API consistency review found no parity issues label Jul 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Parity Review: ✅ Approved

This PR restores cross-repo parity for the OpenAI Chat Completions streaming path.

Change summary: Sets stream_options.include_usage = true on the outbound request before starting a streamed completion, so the OpenAI API emits a final usage chunk that the existing addUsage aggregation can consume. Caller-supplied values are respected and not overridden.

Upstream alignment verified:

  • Python (python/packages/openai/agent_framework_openai/_chat_completion_client.py): Unconditionally sets stream_options: {"include_usage": True} for every streaming request — this PR brings Go to the same default.
  • Python tests (test_openai_chat_completion_client_base.py): Asserts stream_options={"include_usage": True} is present in the outbound call, confirming this is the intended upstream contract.
  • The Go caller-override guard (if !body.StreamOptions.IncludeUsage.Valid()) is a language-idiomatic addition that does not diverge from upstream intent.

No new public Go API surface is introduced. No parity issues found.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by Go API Consistency Review Agent · 33.7 AIC · ⌖ 5.61 AIC · ⊞ 5.7K ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

parity-approved Go API consistency review found no parity issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants