Repository navigation
[release/0.7] RSPEED-3958 Fix Vertex AI failures with parallel tool calls - #3293
Conversation
Multiple tool calls returned from the Vertex api in different chunks would be squashed together resulting in invalid JSON. Add a top-level patch/ directory for fixes to bundled Python packages that can't wait for an upstream release: patches in patch/<package>/NNN-*.patch are applied in order to the installed package in the builder stage of the Containerfile. Add patch/ogx/001-vertexai-number-streamed-tool-calls.patch, a backport of ogx-ai/ogx#6775 to ogx 1.2. (Equivalent of lightspeed-core#3186 for release/0.6.) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
|
🤖 Finished Retro · ✅ Success · Started 4:40 PM UTC · Completed 4:51 PM UTC Commit: Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $5.56 |
Retro: PR #3293 — backport of Vertex AI parallel tool calls fix to release/0.7No new proposals. All findings from this retro are covered by existing open issues. What happenedPR #3293 is a human-authored backport (by owtaylor, with Claude co-authorship) of the Vertex AI parallel tool calls fix from PR #3186 to the No fullsend agents were involved — no code, review, fix, or triage agents ran. CodeRabbit also skipped (auto-reviews disabled on non-default branches). This retro dispatched on merge, making it the second consecutive retro on this same fix (the first ran on PR #3186) with no agent workflow to analyze. Evidence for existing open issues
AssessmentThe workflow outcome was appropriate — quick human approval of a clean backport of an already-reviewed fix is reasonable. The only inefficiency is that retro dispatched at all. Once fullsend-ai/fullsend#939 (or the consolidated design from #5817) ships, this class of unnecessary retro run will be eliminated. |
Description
Equivalent of #3186 for release/0.7 (ogx 1.2.5).
Multiple tool calls returned from the Google Vertex API in different chunks get squashed together into a single call with invalid JSON arguments, because of a bug in ogx's handling of streamed tool calls (ogx-ai/ogx#6774).
This adds
patch/ogx/001-vertexai-number-streamed-tool-calls.patch, which applies the fix from ogx-ai/ogx#6775 in the builder stage of the Containerfile, using thepatch/mechanism introduced in #3186. Since ogx-ai/ogx#6775 was written against ogx main, which shares this code with 1.2.5, the patch is the upstream diff unchanged except for line offsets.Type of change
Tools used to create PR
Identify any AI code assistants used in this PR (for transparency and review context)
Checklist before requesting a review
Testing
--fuzz=0during the build) and tested it against live Vertex AI (gemini-2.5-flash) with a scripted request known to produce parallel tool calls: 30 tries, 13 with parallel calls, 45 tool calls, none malformed. The same image with the unpatchedvertexai.pymounted over the patched one produced merged calls (arguments'{}{}{}') within 5 tries./v1/responsesrequests with more than 20 input items, and request bodies above the previous 128 KiB limit, now up to 1 MiB.🤖 Generated with Claude Code