Skip to content

fix(openai): continue GPT Live only once every backend call is answered - #2495

Merged
longcw merged 1 commit into
mainfrom
longc/gpt-live-session-gate
Sep 15, 2026
Merged

longcw merged 1 commit into
mainfrom
longc/gpt-live-session-gate

Conversation

@longcw

@longcw longcw commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Problem: GPT Live refuses response.create while any function call in the session is unanswered, but the plugin gated each continuation on one response per delegation. A second response.created on the same delegation replaced that record, so an earlier call stopped counting, the early continuation was refused, and the backend never answered again.

Fix: The gate is session-wide: a delegation holds only the calls of the response still running, and confirmed calls wait in one session-wide set. One response.create goes out when nothing runs, nothing is open and an answer went out since the last continuation, and a failed response drops its own calls.

Ports livekit/agents#7266.

Context for reviewing and coding agents

How to see it

holds the continuation until every open call in the conversation is answered runs two responses under one delegation and answers the second call first. On main the plugin sends response.create at that point, so the socket receives three events where the test expects two. The second new test reaches the same state through response.failed.

Why the running response is still tracked per delegation

response.output_item.done carries no response id, so a call is attributed to the newest response.created under its delegation. That attribution is what lets a failed response drop exactly its own calls. The map costs nothing while it holds one entry.

What this port leaves out

The Python fix also warns when a tool result carries reply_required=False, because GPT Live speaks the result anyway. llm.FunctionCallOutput in this repo has no replyRequired field: it lives on _SanitizedOutput in agents/src/voice/generation.ts:253 and never reaches _appendItems, and adding it would change the public API.

Divergence from the Python diff

livekit/agents#7266 moves the call id and function name out of the PII log fields. This repo checks for untagged PII logs, so the lk.pii. tags stay as they are. livekit/agents#7230, the duplicate and incomplete call guard that the Python diff shows as context, is not ported here either.

Blast radius

Only GPTLiveSession in plugins/openai/src/realtime/gpt_live_model.ts changes, and the removed DelegatedResponse interface was private, so pnpm api:check for the plugin reports no API change. The other 80 tests in the file pass without edits.

GPT Live refuses response.create with function_call_outputs_required
while any function call in the session is unanswered, but the plugin
gated each continuation on the one response it tracked per delegation.
A call that fell out of that tracking let a continuation go out early,
the service refused it, and the backend never answered again.

The gate is session-wide: each delegation keeps only the calls of the
response still running, confirmed calls wait in one session-wide set,
and one response.create goes out when nothing runs, nothing is open and
an answer went out since the last continuation. A failed response drops
its calls, since the service discards them and refuses their outputs.

Ports livekit/agents#7266.
@longcw
longcw requested a review from a team as a code owner September 15, 2026 02:05
@changeset-bot

changeset-bot Bot commented Sep 15, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: b7b3be7

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 39 packages
Name Type
@livekit/agents-plugin-openai Patch
@livekit/agents-plugin-anam Patch
@livekit/agents-plugin-cartesia Patch
@livekit/agents-plugin-cerebras Patch
@livekit/agents-plugin-elevenlabs Patch
@livekit/agents-plugin-fishaudio Patch
@livekit/agents-plugin-google Patch
@livekit/agents-plugin-hume Patch
@livekit/agents-plugin-inworld Patch
@livekit/agents-plugin-krisp Patch
@livekit/agents-plugin-neuphonic Patch
@livekit/agents-plugin-perplexity Patch
@livekit/agents-plugin-rime Patch
@livekit/agents-plugin-sarvam Patch
@livekit/agents-plugin-xai Patch
@livekit/agents Patch
@livekit/agents-plugin-anthropic Patch
@livekit/agents-plugin-assemblyai Patch
@livekit/agents-plugin-azure Patch
@livekit/agents-plugin-baseten Patch
@livekit/agents-plugin-bey Patch
@livekit/agents-plugin-deepgram Patch
@livekit/agents-plugin-did Patch
@livekit/agents-plugin-lemonslice Patch
@livekit/agents-plugin-liveavatar Patch
@livekit/agents-plugin-livekit Patch
@livekit/agents-plugin-meta Patch
@livekit/agents-plugin-minimax Patch
@livekit/agents-plugin-mistral Patch
@livekit/agents-plugin-mistralai Patch
@livekit/agents-plugin-phonic Patch
@livekit/agents-plugin-protoface Patch
@livekit/agents-plugin-resemble Patch
@livekit/agents-plugin-runway Patch
@livekit/agents-plugin-silero Patch
@livekit/agents-plugin-soniox Patch
@livekit/agents-plugin-tavus Patch
@livekit/agents-plugins-test Patch
@livekit/agents-plugin-trugen Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@longcw
longcw merged commit ab3dc99 into main Sep 15, 2026
8 checks passed
@longcw
longcw deleted the longc/gpt-live-session-gate branch September 15, 2026 06:03
@github-actions github-actions Bot mentioned this pull request Sep 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants