Repository navigation
Conversation
…mbination When native built-in tools and function declarations are sent in the same request, Gemini 3/4 requires include_server_side_tool_invocations and rejects AUTO function calling mode once the flag is enabled. Opt in only for Gemini 3+ models, use VALIDATED mode for automatic tool choice. Bump google-genai to 1.68.0, the first release exposing the ToolConfig field. Refs AstrBotDevs#9937
Replay the server-side tool_call/tool_response parts (including their thought signatures) returned by Gemini when built-in tools and function calling are combined. The parts are stored on the custom tool call's extra_content so later tool-loop steps can send them back unchanged. Also resolve tool result messages to their function names when rebuilding history (they only carry tool_call_id), and stop re-attaching thought signatures that tool calls or server-side parts already carry. Refs AstrBotDevs#9937
Linked issuesChecked at commit f3ab356 #9937 [Bug] gemini API格式工具调用必定报错 · 2 of 2 met
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 8 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="astrbot/core/provider/sources/gemini_source.py" line_range="506-507" />
<code_context>
+ # carry their own thought signatures. Attaching the same
+ # signature to the narration part would send it twice, so
+ # leave the narration part unsigned and skip it when empty.
+ if text:
+ parts.append(types.Part(text=text))
else:
parts.append(
</code_context>
<issue_to_address>
**Gemini history loses part order**
When a Gemini response interleaves narration, server-side parts, and custom function calls, including across stream chunks, `_process_content_parts` stores server-side parts separately, and `_prepare_conversation` replays them before function calls; `_query_stream` also returns at a function-call chunk without collecting later chunks. The next request therefore sends reordered or incomplete tool history, which Gemini can reject or mis-handle.
Preserve the original order of narration, server-side parts, and function calls, and collect all relevant parts before returning a streamed response.
Also at `astrbot/core/provider/sources/gemini_source.py:527-538`, `astrbot/core/provider/sources/gemini_source.py:722-743`, `astrbot/core/provider/sources/gemini_source.py:931`, `astrbot/core/provider/sources/gemini_source.py:971`.
</issue_to_address>
### Comment 2
<location path="astrbot/core/provider/sources/gemini_source.py" line_range="735" />
<code_context>
+ or getattr(part, "executable_code", None)
+ or getattr(part, "code_execution_result", None)
+ ]
+ if server_side_parts and llm_response.tools_call_ids:
+ google_extra = llm_response.tools_call_extra_content.setdefault(
+ llm_response.tools_call_ids[0], {}
</code_context>
<issue_to_address>
**Built-in tool history disappears**
When gemini returns server-side tool parts without a custom function call and the conversation continues, `_process_content_parts` and the streaming path attach server-side parts only to custom tool calls, while the runner stores only assistant text when there is no custom call. The next Gemini request omits the built-in tool history and can fail validation or lose tool context.
Persist server-side parts in the response history even when Gemini returns no custom function call.
Also at `astrbot/core/provider/sources/gemini_source.py:940-941`, `astrbot/core/provider/sources/gemini_source.py:971-978`.
</issue_to_address>
### Comment 3
<location path="astrbot/core/provider/sources/gemini_source.py" line_range="501" />
<code_context>
- pass
+ for tool in message.get("tool_calls") or []
+ )
+ if thinking_signature and has_tool_thought_signature:
+ # Tool calls and replayed server-side tool parts already
+ # carry their own thought signatures. Attaching the same
</code_context>
<issue_to_address>
**Narration signature is omitted**
When the narration signature differs from a signature on a tool or server-side part, `_prepare_conversation` treats any tool or server-side signature as a duplicate without comparing values, so it omits the narration signature. Gemini receives incomplete signed history on the next request.
Suppress the narration signature only when a tool or server-side part carries the same signature.
</issue_to_address>
### Comment 4
<location path="astrbot/core/provider/sources/gemini_source.py" line_range="736-740" />
<code_context>
+ or getattr(part, "code_execution_result", None)
+ ]
+ if server_side_parts and llm_response.tools_call_ids:
+ google_extra = llm_response.tools_call_extra_content.setdefault(
+ llm_response.tools_call_ids[0], {}
+ ).setdefault("google", {})
+ google_extra["server_parts"] = [
+ *server_side_parts,
+ *(google_extra.get("server_parts") or []),
+ ]
</code_context>
<issue_to_address>
**Requery loses server-side context**
When the `skills_like` path re-queries and returns different tool-call IDs, `ToolLoopAgentRunner._step` replaces the tool-call IDs without transferring `tools_call_extra_content`, leaving the saved server-side parts under the original IDs. The next Gemini request omits that context.
Transfer the saved `tools_call_extra_content` to the re-query's tool-call IDs.
Also at `astrbot/core/provider/sources/gemini_source.py:735`.
</issue_to_address>
### Comment 5
<location path="astrbot/core/provider/sources/gemini_source.py" line_range="735-743" />
<code_context>
+ or getattr(part, "executable_code", None)
+ or getattr(part, "code_execution_result", None)
+ ]
+ if server_side_parts and llm_response.tools_call_ids:
+ google_extra = llm_response.tools_call_extra_content.setdefault(
+ llm_response.tools_call_ids[0], {}
+ ).setdefault("google", {})
+ google_extra["server_parts"] = [
+ *server_side_parts,
+ *(google_extra.get("server_parts") or []),
+ ]
chain_result = MessageChain(chain=chain)
llm_response.result_chain = chain_result
if validate_output:
</code_context>
<issue_to_address>
**Skills requery drops server context**
When skills-like tool schema mode is enabled and Gemini returns server-side parts with a custom tool call, `_resolve_tool_exec` builds its parameter requery from `run_context.messages`, which does not yet contain the current response, then replaces or rewrites that response’s tool calls. The requery therefore omits the captured server-side parts, and the later tool-loop request loses them too.
Include the current response’s server-side parts in the parameter requery context and preserve them under the resulting tool-call ID.
</issue_to_address>
### Comment 6
<location path="tests/test_gemini_source.py" line_range="677" />
<code_context>
+ assert parts[0].tool_call.id == "search-1"
+ assert parts[0].thought_signature == b"\x01\x02"
+ assert parts[1].tool_response is not None
+ assert parts[2].function_call is not None
+ assert parts[2].function_call.name == "generate_image"
+ assert parts[2].thought_signature == b"\x03"
</code_context>
<issue_to_address>
**Lost tool payloads go unnoticed**
When replayed server-side parts retain their wrapper but lose payload fields, when replay serialization drops a server-side part’s fields but leaves its wrapper present, `test_gemini_prepare_conversation_replays_server_side_tool_parts` passes because it checks only that `tool_response` is non-`None`; the streaming test checks only key presence and order, so lost IDs or response data go unnoticed.
Assert that replayed and streamed server-side parts preserve their IDs, tool types, arguments, responses, and thought signatures.
Also at `tests/test_gemini_source.py:708`.
</issue_to_address>
### Comment 7
<location path="tests/test_gemini_source.py" line_range="20" />
<code_context>
+def _gemini_config_provider(
+ model: str, provider_config: dict | None = None
+) -> ProviderGoogleGenAI:
+ """Build a provider stub that can build request configs without network I/O."""
+ provider = ProviderGoogleGenAI.__new__(ProviderGoogleGenAI)
+ provider.provider_config = provider_config or {}
</code_context>
<issue_to_address>
**New docstrings omit required sections**
When an added docstring documents a function with parameters or a return value, the added docstrings for helpers and parameterized tests describe functions with parameters or return values without Google-format `Args:` or `Returns:` sections, contrary to the repository's docstring rule.
Add the appropriate Google-format `Args:` and `Returns:` sections to the affected docstrings.
Also at `tests/test_gemini_source.py:31`, `tests/test_gemini_source.py:95`, `tests/test_gemini_source.py:128`, `tests/test_gemini_source.py:197`, `tests/test_gemini_source.py:572`, `tests/test_gemini_source.py:754`.
</issue_to_address>
### Comment 8
<location path="tests/test_gemini_source.py" line_range="17-46" />
<code_context>
from astrbot.core.provider.sources.gemini_source import ProviderGoogleGenAI
+def _gemini_config_provider(
+ model: str, provider_config: dict | None = None
+) -> ProviderGoogleGenAI:
+ """Build a provider stub that can build request configs without network I/O."""
+ provider = ProviderGoogleGenAI.__new__(ProviderGoogleGenAI)
+ provider.provider_config = provider_config or {}
+ provider.provider_settings = {}
+ provider.model_name = model
+ provider.safety_settings = []
+ return provider
+
+
+def _gemini_func_toolset() -> SimpleNamespace:
</code_context>
<issue_to_address>
**Tests added against repository rule**
When the PR adds new test code to `tests/test_gemini_source.py`, the added test helpers and regression tests expand the test suite despite the repository instruction to avoid generating test files.
Remove the added test code to comply with the repository instruction.
Also at `tests/test_gemini_source.py:570-810`.
</issue_to_address>Sourcery assessment
Needs a human reviewer. 6 findings to address first, and the change stores Gemini server-side tool parts and signatures in tool-call conversation history so later requests can replay them. If that data is recorded incorrectly, follow-up requests for affected conversations may fail; reverting stops new entries but does not remove existing ones, which can be repaired by clearing those conversations.
Blocking findings: astrbot/core/provider/sources/gemini_source.py:507, astrbot/core/provider/sources/gemini_source.py:735, astrbot/core/provider/sources/gemini_source.py:501, astrbot/core/provider/sources/gemini_source.py:740, astrbot/core/provider/sources/gemini_source.py:743, and 1 more
| if text: | ||
| parts.append(types.Part(text=text)) |
There was a problem hiding this comment.
🟡 Medium · Gemini history loses part order
When a Gemini response interleaves narration, server-side parts, and custom function calls, including across stream chunks, _process_content_parts stores server-side parts separately, and _prepare_conversation replays them before function calls; _query_stream also returns at a function-call chunk without collecting later chunks. The next request therefore sends reordered or incomplete tool history, which Gemini can reject or mis-handle.
Preserve the original order of narration, server-side parts, and function calls, and collect all relevant parts before returning a streamed response.
Also at astrbot/core/provider/sources/gemini_source.py:527-538, astrbot/core/provider/sources/gemini_source.py:722-743, astrbot/core/provider/sources/gemini_source.py:931, astrbot/core/provider/sources/gemini_source.py:971.
Prompt for AI agents
In `astrbot/core/provider/sources/gemini_source.py` at lines 506-507:
**Gemini history loses part order**
When a Gemini response interleaves narration, server-side parts, and custom function calls, including across stream chunks, `_process_content_parts` stores server-side parts separately, and `_prepare_conversation` replays them before function calls; `_query_stream` also returns at a function-call chunk without collecting later chunks. The next request therefore sends reordered or incomplete tool history, which Gemini can reject or mis-handle.
Preserve the original order of narration, server-side parts, and function calls, and collect all relevant parts before returning a streamed response.
Also at `astrbot/core/provider/sources/gemini_source.py:527-538`, `astrbot/core/provider/sources/gemini_source.py:722-743`, `astrbot/core/provider/sources/gemini_source.py:931`, `astrbot/core/provider/sources/gemini_source.py:971`.| or getattr(part, "executable_code", None) | ||
| or getattr(part, "code_execution_result", None) | ||
| ] | ||
| if server_side_parts and llm_response.tools_call_ids: |
There was a problem hiding this comment.
🟠 High · Built-in tool history disappears
When gemini returns server-side tool parts without a custom function call and the conversation continues, _process_content_parts and the streaming path attach server-side parts only to custom tool calls, while the runner stores only assistant text when there is no custom call. The next Gemini request omits the built-in tool history and can fail validation or lose tool context.
Persist server-side parts in the response history even when Gemini returns no custom function call.
Also at astrbot/core/provider/sources/gemini_source.py:940-941, astrbot/core/provider/sources/gemini_source.py:971-978.
Prompt for AI agents
In `astrbot/core/provider/sources/gemini_source.py` at line 735:
**Built-in tool history disappears**
When gemini returns server-side tool parts without a custom function call and the conversation continues, `_process_content_parts` and the streaming path attach server-side parts only to custom tool calls, while the runner stores only assistant text when there is no custom call. The next Gemini request omits the built-in tool history and can fail validation or lose tool context.
Persist server-side parts in the response history even when Gemini returns no custom function call.
Also at `astrbot/core/provider/sources/gemini_source.py:940-941`, `astrbot/core/provider/sources/gemini_source.py:971-978`.| google_extra = llm_response.tools_call_extra_content.setdefault( | ||
| llm_response.tools_call_ids[0], {} | ||
| ).setdefault("google", {}) | ||
| google_extra["server_parts"] = [ | ||
| *server_side_parts, |
There was a problem hiding this comment.
🟡 Medium · Requery loses server-side context
When the skills_like path re-queries and returns different tool-call IDs, ToolLoopAgentRunner._step replaces the tool-call IDs without transferring tools_call_extra_content, leaving the saved server-side parts under the original IDs. The next Gemini request omits that context.
Transfer the saved tools_call_extra_content to the re-query's tool-call IDs.
Also at astrbot/core/provider/sources/gemini_source.py:735.
Prompt for AI agents
In `astrbot/core/provider/sources/gemini_source.py` at lines 736-740:
**Requery loses server-side context**
When the `skills_like` path re-queries and returns different tool-call IDs, `ToolLoopAgentRunner._step` replaces the tool-call IDs without transferring `tools_call_extra_content`, leaving the saved server-side parts under the original IDs. The next Gemini request omits that context.
Transfer the saved `tools_call_extra_content` to the re-query's tool-call IDs.
Also at `astrbot/core/provider/sources/gemini_source.py:735`.| if server_side_parts and llm_response.tools_call_ids: | ||
| google_extra = llm_response.tools_call_extra_content.setdefault( | ||
| llm_response.tools_call_ids[0], {} | ||
| ).setdefault("google", {}) | ||
| google_extra["server_parts"] = [ | ||
| *server_side_parts, | ||
| *(google_extra.get("server_parts") or []), | ||
| ] | ||
| chain_result = MessageChain(chain=chain) |
There was a problem hiding this comment.
🟡 Medium · Skills requery drops server context
When skills-like tool schema mode is enabled and Gemini returns server-side parts with a custom tool call, _resolve_tool_exec builds its parameter requery from run_context.messages, which does not yet contain the current response, then replaces or rewrites that response’s tool calls. The requery therefore omits the captured server-side parts, and the later tool-loop request loses them too.
Include the current response’s server-side parts in the parameter requery context and preserve them under the resulting tool-call ID.
Prompt for AI agents
In `astrbot/core/provider/sources/gemini_source.py` at lines 735-743:
**Skills requery drops server context**
When skills-like tool schema mode is enabled and Gemini returns server-side parts with a custom tool call, `_resolve_tool_exec` builds its parameter requery from `run_context.messages`, which does not yet contain the current response, then replaces or rewrites that response’s tool calls. The requery therefore omits the captured server-side parts, and the later tool-loop request loses them too.
Include the current response’s server-side parts in the parameter requery context and preserve them under the resulting tool-call ID.| def _gemini_config_provider( | ||
| model: str, provider_config: dict | None = None | ||
| ) -> ProviderGoogleGenAI: | ||
| """Build a provider stub that can build request configs without network I/O.""" | ||
| provider = ProviderGoogleGenAI.__new__(ProviderGoogleGenAI) | ||
| provider.provider_config = provider_config or {} | ||
| provider.provider_settings = {} | ||
| provider.model_name = model | ||
| provider.safety_settings = [] | ||
| return provider | ||
|
|
||
|
|
||
| def _gemini_func_toolset() -> SimpleNamespace: | ||
| """Minimal ToolSet stand-in exposing one function declaration.""" | ||
| return SimpleNamespace( | ||
| get_func_desc_google_genai_style=lambda: { | ||
| "function_declarations": [ | ||
| { | ||
| "name": "generate_image", | ||
| "description": "Generate an image", | ||
| "parameters": {"type": "object", "properties": {}}, | ||
| } | ||
| ] | ||
| } | ||
| ) | ||
|
|
||
|
|
||
| @pytest.mark.asyncio | ||
| async def test_gemini_thinking_level_is_serialized_on_every_request(): | ||
| model = "gemini-3.7-flash" |
There was a problem hiding this comment.
⚪ Low · Tests added against repository rule
When the PR adds new test code to tests/test_gemini_source.py, the added test helpers and regression tests expand the test suite despite the repository instruction to avoid generating test files.
Remove the added test code to comply with the repository instruction.
Also at tests/test_gemini_source.py:570-810.
Prompt for AI agents
In `tests/test_gemini_source.py` at lines 17-210:
**Tests added against repository rule**
When the PR adds new test code to `tests/test_gemini_source.py`, the added test helpers and regression tests expand the test suite despite the repository instruction to avoid generating test files.
Remove the added test code to comply with the repository instruction.
Also at `tests/test_gemini_source.py:570-810`.From .github/copilot-instructions.md L27:
Do not generate test files for now.
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
@sourcery-ai review please |
There was a problem hiding this comment.
Hey - I've found 5 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="astrbot/core/provider/sources/gemini_source.py" line_range="728" />
<code_context>
+ or getattr(part, "executable_code", None)
+ or getattr(part, "code_execution_result", None)
+ ]
+ if server_side_parts and llm_response.tools_call_ids:
+ google_extra = llm_response.tools_call_extra_content.setdefault(
+ llm_response.tools_call_ids[0], {}
</code_context>
<issue_to_address>
**Built-in tool history disappears**
When gemini uses a built-in tool but returns no custom function call, and the conversation continues, `_process_content_parts` does not save `server_side_parts` without a tool-call ID, and the streaming path likewise leaves `pending_server_side_parts` unattached. The agent records only the assistant text and reasoning, so a later request omits the built-in tool history and Gemini rejects or mishandles the conversation.
Persist and replay server-side tool parts even when Gemini returns no custom function call.
Also at `astrbot/core/provider/sources/gemini_source.py:463`, `astrbot/core/provider/sources/gemini_source.py:934`, `astrbot/core/provider/sources/gemini_source.py:964-971`.
</issue_to_address>
### Comment 2
<location path="astrbot/core/provider/sources/gemini_source.py" line_range="475-499" />
<code_context>
+ # invocations are enabled, the API requires them to be
+ # sent back unchanged, including their thought signatures.
+ for server_side_part in google_extra.get("server_parts") or []:
+ try:
+ tool_parts.append(
+ types.Part.model_validate(server_side_part)
</code_context>
<issue_to_address>
**Tool history order changes**
When gemini returns server-side parts before or interleaved with narration or custom function calls, `_process_content_parts` separates server-side parts from their original positions, and `_prepare_conversation` replays them before the associated function calls. Gemini receives tool history in a different order and can reject or incorrectly process the follow-up request.
Preserve the original positions of server-side parts and replay them in that order.
</issue_to_address>
### Comment 3
<location path="astrbot/core/provider/sources/gemini_source.py" line_range="925" />
<code_context>
+ chunk_parts = chunk.candidates[0].content.parts or []
+
+ if chunk_parts and any(part.function_call for part in chunk_parts):
llm_response = LLMResponse("assistant", is_chunk=False)
llm_response.raw_completion = chunk
llm_response.result_chain = self._process_content_parts(
</code_context>
<issue_to_address>
**Later tool context is dropped**
When gemini emits a server-side tool part in a stream chunk after the first custom function-call chunk, `_query_stream` returns as soon as it sees a custom function call, so server-side tool parts arriving in subsequent chunks are never added to `pending_server_side_parts` or attached to the tool call. `_prepare_conversation` then replays incomplete history, and the next combined-tool request can fail.
Continue consuming the stream and accumulate server-side parts before yielding the completed tool-call response.
</issue_to_address>
### Comment 4
<location path="astrbot/core/provider/sources/gemini_source.py" line_range="728" />
<code_context>
+ or getattr(part, "executable_code", None)
+ or getattr(part, "code_execution_result", None)
+ ]
+ if server_side_parts and llm_response.tools_call_ids:
+ google_extra = llm_response.tools_call_extra_content.setdefault(
+ llm_response.tools_call_ids[0], {}
</code_context>
<issue_to_address>
**Re-query loses server-side history**
When the agent uses `skills_like` mode and the initial Gemini response includes server-side parts alongside a custom function call, `ToolLoopAgentRunner._resolve_tool_exec` rebuilds the re-query context from pre-call messages and replaces the original response, so the re-query response and later tool-loop history omit the original `server_parts`. Gemini receives a follow-up request without the required tool history and rejects or misprocesses it.
Carry the original response's server-side parts into the `skills_like` re-query context and its replacement response.
Also at `astrbot/core/provider/sources/gemini_source.py:729-735`.
</issue_to_address>
### Comment 5
<location path="astrbot/core/provider/sources/gemini_source.py" line_range="732-734" />
<code_context>
+ google_extra = llm_response.tools_call_extra_content.setdefault(
+ llm_response.tools_call_ids[0], {}
+ ).setdefault("google", {})
+ google_extra["server_parts"] = [
+ *server_side_parts,
+ *(google_extra.get("server_parts") or []),
+ ]
chain_result = MessageChain(chain=chain)
</code_context>
<issue_to_address>
**Server context disappears after re-query**
When a Gemini request with combined built-in and function tools runs with `tool_schema_mode="skills_like"` and takes the re-query branch, `ToolLoopAgentRunner._step` replaces the tool-call IDs after a `skills_like` re-query without transferring `tools_call_extra_content`. `to_openai_tool_calls_model` then looks up metadata using the new IDs, so `_prepare_conversation` omits the stored server-side parts and the next Gemini request loses required tool context.
Preserve or merge the original and re-query `tools_call_extra_content` under the IDs that will be serialized into history.
</issue_to_address>Sourcery assessment
Approval pending. 5 findings to address first.
Blocking findings: astrbot/core/provider/sources/gemini_source.py:728, astrbot/core/provider/sources/gemini_source.py:499, astrbot/core/provider/sources/gemini_source.py:925, astrbot/core/provider/sources/gemini_source.py:728, astrbot/core/provider/sources/gemini_source.py:734
| or getattr(part, "executable_code", None) | ||
| or getattr(part, "code_execution_result", None) | ||
| ] | ||
| if server_side_parts and llm_response.tools_call_ids: |
There was a problem hiding this comment.
🟡 Medium · Built-in tool history disappears
When gemini uses a built-in tool but returns no custom function call, and the conversation continues, _process_content_parts does not save server_side_parts without a tool-call ID, and the streaming path likewise leaves pending_server_side_parts unattached. The agent records only the assistant text and reasoning, so a later request omits the built-in tool history and Gemini rejects or mishandles the conversation.
Persist and replay server-side tool parts even when Gemini returns no custom function call.
Also at astrbot/core/provider/sources/gemini_source.py:463, astrbot/core/provider/sources/gemini_source.py:934, astrbot/core/provider/sources/gemini_source.py:964-971.
Prompt for AI agents
In `astrbot/core/provider/sources/gemini_source.py` at line 728:
**Built-in tool history disappears**
When gemini uses a built-in tool but returns no custom function call, and the conversation continues, `_process_content_parts` does not save `server_side_parts` without a tool-call ID, and the streaming path likewise leaves `pending_server_side_parts` unattached. The agent records only the assistant text and reasoning, so a later request omits the built-in tool history and Gemini rejects or mishandles the conversation.
Persist and replay server-side tool parts even when Gemini returns no custom function call.
Also at `astrbot/core/provider/sources/gemini_source.py:463`, `astrbot/core/provider/sources/gemini_source.py:934`, `astrbot/core/provider/sources/gemini_source.py:964-971`.| try: | ||
| tool_parts.append( | ||
| types.Part.model_validate(server_side_part) | ||
| ) | ||
| except Exception as e: | ||
| logger.warning( | ||
| "Failed to restore Gemini server-side tool part: %s", | ||
| e, | ||
| exc_info=True, | ||
| ) | ||
| part = types.Part.from_function_call( | ||
| name=function_name, | ||
| args=json.loads(tool["function"]["arguments"]), | ||
| ) | ||
| # we should set thought_signature back to part if exists | ||
| # for more info about thought_signature, see: | ||
| # https://ai.google.dev/gemini-api/docs/thought-signatures | ||
| ts_bs64 = google_extra.get("thought_signature") | ||
| if ts_bs64: | ||
| part.thought_signature = base64.b64decode(ts_bs64) | ||
| tool_parts.append(part) | ||
|
|
||
| if isinstance(content, str): | ||
| parts.append(types.Part.from_text(text=content)) | ||
| elif isinstance(content, list): |
There was a problem hiding this comment.
🟡 Medium · Tool history order changes
When gemini returns server-side parts before or interleaved with narration or custom function calls, _process_content_parts separates server-side parts from their original positions, and _prepare_conversation replays them before the associated function calls. Gemini receives tool history in a different order and can reject or incorrectly process the follow-up request.
Preserve the original positions of server-side parts and replay them in that order.
Prompt for AI agents
In `astrbot/core/provider/sources/gemini_source.py` at lines 475-541:
**Tool history order changes**
When gemini returns server-side parts before or interleaved with narration or custom function calls, `_process_content_parts` separates server-side parts from their original positions, and `_prepare_conversation` replays them before the associated function calls. Gemini receives tool history in a different order and can reject or incorrectly process the follow-up request.
Preserve the original positions of server-side parts and replay them in that order.| chunk_parts = chunk.candidates[0].content.parts or [] | ||
|
|
||
| if chunk_parts and any(part.function_call for part in chunk_parts): | ||
| llm_response = LLMResponse("assistant", is_chunk=False) |
There was a problem hiding this comment.
🟠 High · Later tool context is dropped
When gemini emits a server-side tool part in a stream chunk after the first custom function-call chunk, _query_stream returns as soon as it sees a custom function call, so server-side tool parts arriving in subsequent chunks are never added to pending_server_side_parts or attached to the tool call. _prepare_conversation then replays incomplete history, and the next combined-tool request can fail.
Continue consuming the stream and accumulate server-side parts before yielding the completed tool-call response.
Prompt for AI agents
In `astrbot/core/provider/sources/gemini_source.py` at line 925:
**Later tool context is dropped**
When gemini emits a server-side tool part in a stream chunk after the first custom function-call chunk, `_query_stream` returns as soon as it sees a custom function call, so server-side tool parts arriving in subsequent chunks are never added to `pending_server_side_parts` or attached to the tool call. `_prepare_conversation` then replays incomplete history, and the next combined-tool request can fail.
Continue consuming the stream and accumulate server-side parts before yielding the completed tool-call response.| or getattr(part, "executable_code", None) | ||
| or getattr(part, "code_execution_result", None) | ||
| ] | ||
| if server_side_parts and llm_response.tools_call_ids: |
There was a problem hiding this comment.
🟡 Medium · Re-query loses server-side history
When the agent uses skills_like mode and the initial Gemini response includes server-side parts alongside a custom function call, ToolLoopAgentRunner._resolve_tool_exec rebuilds the re-query context from pre-call messages and replaces the original response, so the re-query response and later tool-loop history omit the original server_parts. Gemini receives a follow-up request without the required tool history and rejects or misprocesses it.
Carry the original response's server-side parts into the skills_like re-query context and its replacement response.
Also at astrbot/core/provider/sources/gemini_source.py:729-735.
Prompt for AI agents
In `astrbot/core/provider/sources/gemini_source.py` at line 728:
**Re-query loses server-side history**
When the agent uses `skills_like` mode and the initial Gemini response includes server-side parts alongside a custom function call, `ToolLoopAgentRunner._resolve_tool_exec` rebuilds the re-query context from pre-call messages and replaces the original response, so the re-query response and later tool-loop history omit the original `server_parts`. Gemini receives a follow-up request without the required tool history and rejects or misprocesses it.
Carry the original response's server-side parts into the `skills_like` re-query context and its replacement response.
Also at `astrbot/core/provider/sources/gemini_source.py:729-735`.| google_extra["server_parts"] = [ | ||
| *server_side_parts, | ||
| *(google_extra.get("server_parts") or []), |
There was a problem hiding this comment.
🟡 Medium · Server context disappears after re-query
When a Gemini request with combined built-in and function tools runs with tool_schema_mode="skills_like" and takes the re-query branch, ToolLoopAgentRunner._step replaces the tool-call IDs after a skills_like re-query without transferring tools_call_extra_content. to_openai_tool_calls_model then looks up metadata using the new IDs, so _prepare_conversation omits the stored server-side parts and the next Gemini request loses required tool context.
Preserve or merge the original and re-query tools_call_extra_content under the IDs that will be serialized into history.
Prompt for AI agents
In `astrbot/core/provider/sources/gemini_source.py` at lines 732-734:
**Server context disappears after re-query**
When a Gemini request with combined built-in and function tools runs with `tool_schema_mode="skills_like"` and takes the re-query branch, `ToolLoopAgentRunner._step` replaces the tool-call IDs after a `skills_like` re-query without transferring `tools_call_extra_content`. `to_openai_tool_calls_model` then looks up metadata using the new IDs, so `_prepare_conversation` omits the stored server-side parts and the next Gemini request loses required tool context.
Preserve or merge the original and re-query `tools_call_extra_content` under the IDs that will be serialized into history.
Fixes #9937.
Motivation
Since #8833 allowed Gemini built-in tools (
gm_native_search,gm_native_coderunner,gm_url_context) and custom function declarations (function_declarations) to coexist in the samegenerateContentrequest, enabling any built-in tool alongside function tools (whether from a third-party plugin or built-in AstrBot tools such asweb_search_exaandastrbot_execute_python) on Gemini 3/4 models fails with two cascading400 INVALID_ARGUMENTerrors reported in #9937:include_server_side_tool_invocationsnot set):functionResponse.namepopulated withtool_call_idinstead of function name, and server-side tool invocation parts dropped):Root Cause
ToolConfigopt-in & incompatible calling mode: Combining Gemini built-in tools withfunction_declarationsrequirestool_config.include_server_side_tool_invocations = TrueandFunctionCallingConfigMode.VALIDATED(AUTOis rejected when server-side tool invocations are enabled). This capability is only supported on Gemini 3/4 models (Gemini 1.x/2.x reject the flag) and requiresgoogle-genai>=1.68.0for theToolConfig.include_server_side_tool_invocationsfield andPart.tool_call/Part.tool_responsetypes.include_server_side_tool_invocations=True, the model response includes server-sidetool_call/tool_response(andexecutable_code/code_execution_result) parts withid,tool_type, andthought_signaturethat must be replayed in subsequent turns of the tool loop. Previously,_process_content_partsand_query_streamdiscarded these parts, and_prepare_conversationcould also duplicatethought_signatureonto accompanying narration text parts.functionResponse.namemismatch on tool results: When server-side tool invocations are enabled, Gemini populatespart.function_call.id(e.g.,call_513417). AstrBot stores that ID intool_call_idonrole="tool"messages without a"name"field, causing_prepare_conversationto fall back tomessage.get("name", message["tool_call_id"])and send the call ID instead of the function name inFunctionResponse.name.Related Issues & PRs
AUTO→VALIDATED, gate legacy models, bumpgoogle-genai, or replay server-side tool parts / mapfunctionResponse.name.functionResponse.namemapping PR; covered by thetool_id_to_namemapping in this PR.thought_signaturein cross-provider histories (orthogonal).functionResponse.idround-tripping required by certain proxy backends; independent follow-up not claimed by this PR._prepare_query_config(non-overlapping hunks).Modifications / 改动点
astrbot/core/provider/sources/gemini_source.py:_prepare_query_config(L263–295): Detect built-in tools (google_search,code_execution,url_context) whenfunction_declarationsare present. Enablecombines_toolswhenhas_builtin_toolis true andmodel_name.lower()contains"gemini-3"or"gemini-4"(substring matching supports gateway/relay prefix aliases such as[抗截断]gemini-3.8-flashorgateway/gemini-3.8-flash-high). SetFunctionCallingConfigMode.ANYwhentool_choice == "required",VALIDATEDwhencombines_toolsis true, andAUTOotherwise; setToolConfig(include_server_side_tool_invocations=True)only whencombines_toolsis true._prepare_conversation—tool_id_to_namemapping (L439–442, L467–469, L548–564): Recordtool["id"] -> tool["function"]["name"]while iterating assistanttool_calls, and resolverole="tool"FunctionResponse.nameviamessage.get("name") or tool_id_to_name.get(tool_call_id, tool_call_id)._prepare_conversation— server-side part replay (L462–495, L541): Reconstructextra_content.google.server_partsviatypes.Part.model_validate(...)intotool_partsbefore appendingfunction_callparts and restoringthought_signature._prepare_conversation—thought_signaturededuplication (L519–540): Compare decodedthinking_signaturebytes againsttool_part.thought_signatureacrosstool_parts(has_duplicate_thought_signature, covering both customfunction_calland replayedserver_parts) so narration text parts omit duplicate signatures (and skip empty narration parts) while preserving distinct narration signatures._process_content_parts(L715–735) &_query_stream(L906–971): Capture server-side tool parts (tool_call,tool_response,executable_code,code_execution_result) viapart.model_dump(exclude_none=True, mode="json")(accumulating across stream chunks viapending_server_side_parts) and attach them to the first custom tool call'sextra_content.google.server_parts.pyproject.toml(L29) &requirements.txt(L18):google-genaiversion to>=1.68.0(first release providingToolConfig.include_server_side_tool_invocationsandPart.tool_call/Part.tool_response).tests/test_gemini_source.py:VALIDATED+include_server_side_tool_invocations=Trueopt-in, model alias / case-insensitive / Gemini 4 matching,tool_choice="required"(ANYmode preservation), function-only (AUTO), built-in-only (tool_config=None), legacy Gemini 2.x exclusion, non-streaming and streaming server-side part capture & replay (with full field-level assertions and malformed-part fallback coverage),functionResponse.nameresolution fromtool_call_id, andthought_signaturededuplication / distinct signature preservation with empty and non-empty narration.Behavior Matrix
function_calling_config.modeinclude_server_side_tool_invocationsAUTO(orANYifrequired)Nonetool_config = NoneNoneAUTO(orANYifrequired)Nonetool_choicedefault)VALIDATEDTruetool_choice="required")ANYTruegoogle-genaifloor to>=1.68.0only affects environments with an older SDK installed.Screenshots or Test Results / 运行截图或测试结果
1. Automated Tests & Linting
2. End-to-End Runtime Verification (Official Gemini API)
google_geminiprovider configured withgemini-3.8-flashagainst the official Gemini API endpoint (generativelanguage.googleapis.com),gm_native_searchenabled, and a third-party image generation plugin registering thegenerate_imagefunction tool.400 INVALID_ARGUMENTon turn 1; if only the flag is enabled, turn 2 fails with400 INVALID_ARGUMENTdue tofunctionResponse.namemismatch):Checklist / 检查清单
😊 If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
/ 如果 PR 中有新加入的功能，已经通过 Issue / 邮件等方式和作者讨论过。 (Bug fix only, no new features; tracked in [Bug] gemini API格式工具调用必定报错 #9937, related to fix(provider): opt in server-side tool invocations for gemini built-in tools #9978 / fix: preserve Gemini tool response function names #10174 / fix(dashboard): correct outdated Gemini native tool hints #10875.)
👀 My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above.
/ 我的更改经过了良好的测试，并已在上方提供了“验证步骤”和“运行截图”。
📚 I checked the affected WebUI instructions and screenshots in
docs/zhanddocs/enagainst the changed navigation, page structure, and labels, and updated them in this PR (or explained why no documentation update is needed). For renamed, moved, or merged entry points, I included an old entry → new entry mapping in the documentation and changelog./ 我已对照变化后的 WebUI 入口、页面结构和术语，核对并在本 PR 中更新
docs/zh和docs/en的相关操作说明与截图（或说明无需更新文档的原因）。入口改名、移动或合并时，已在文档和 changelog 中补充 旧入口 → 新入口 对照。 (No WebUI navigation, page structure, or terminology changes; related config hint text was already updated in fix(dashboard): correct outdated Gemini native tool hints #10875.)🤓 I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations in
requirements.txtandpyproject.toml./ 我确保没有引入新依赖库，或者引入了新依赖库的同时将其添加到
requirements.txt和pyproject.toml文件相应位置。 (No new dependencies added; bumped existinggoogle-genaifloor to>=1.68.0in bothpyproject.tomlandrequirements.txt.)😮 My changes do not introduce malicious code.
/ 我的更改没有引入恶意代码。
Summary by Sourcery
Enable reliable Gemini 3/4 built-in tool and function-calling combinations across multi-turn tool interactions.
Bug Fixes:
Enhancements:
Build:
Tests: