Repository navigation
[Fix] Reasoning with tool use fails on OpenAI Compatible endpoints that require the Responses API - #2498
[Fix] Reasoning with tool use fails on OpenAI Compatible endpoints that require the Responses API#2498zoomote2[bot] wants to merge 1 commit into
Conversation
…ible provider
Reasoning models such as GPT-6.1 Terra on Azure OpenAI reject
reasoning_effort combined with function tools on /chat/completions and
require /v1/responses. The OpenAI Compatible provider only spoke Chat
Completions, so reasoning could not be combined with tool use on those
endpoints.
Add an openAiUseResponsesApi setting. When enabled, createMessage posts
to {baseUrl}/responses with the provider's custom base URL, custom
model/deployment ID, and custom headers. The resolved reasoning_effort
is translated to the Responses reasoning.effort shape, tools use the
flattened Responses function shape, and the shared Responses input and
stream transforms (also used by the xAI provider) handle conversion.
Chat Completions stays the default, so existing setups are unchanged.
|
Important Review skippedAuto reviews are limited based on label configuration. 🏷️ Required labels (at least one): 1
Please check the settings in the CodeRabbit UI or the
|
|
5 issues outstanding. See session
Reviewed 05e80b1 |
Review statusThis PR was opened by an automated account. A human maintainer must verify the change intent, provenance, and validation before merging. Current step: Fix the failing required CI checks; awaiting-maintainer requires CI and automated review completion. Review-state labels are managed by this workflow; do not edit them manually. |
| model: modelId, | ||
| instructions: systemPrompt, | ||
| input: convertToResponsesApiInput(messages), | ||
| stream: true, |
There was a problem hiding this comment.
When openAiUseResponsesApi is combined with openAiStreamingEnabled: false, this still sends stream: true, so the existing Enable streaming setting is silently ignored. Please honor the non-streaming configuration on this path (and cover that combination), or prevent the unsupported combination explicitly.
| try { | ||
| let stream: AsyncIterable<OpenAI.Responses.ResponseStreamEvent> | ||
| try { | ||
| stream = await this.client.responses.create( |
There was a problem hiding this comment.
This request never passes through withExtraBody, unlike every Chat Completions request in this provider. Enabling Responses API therefore silently drops configured Extra Body fields such as service_tier or gateway metadata. Merge this.extraBody into the Responses request with the existing request-owned-field precedence and add a regression test.
|
|
||
| const requestBody: OpenAI.Responses.ResponseCreateParamsStreaming = { | ||
| model: modelId, | ||
| instructions: systemPrompt, |
There was a problem hiding this comment.
The existing OpenAI-compatible paths sanitize lone UTF-16 surrogates in the system prompt and converted message history, but this path forwards both unchanged (convertToResponsesApiInput does not sanitize them). A task containing a sliced surrogate can therefore be rejected again when Responses API is enabled. Please preserve the existing sanitization invariant for this request body and add a Responses-path test.
| {t("settings:providers.openAiStrictToolSchemasDescription")} | ||
| </div> | ||
| </div> | ||
| <div> |
There was a problem hiding this comment.
This visible layout change needs updated committed Playwright baselines under webview-ui/AGENTS.md. The current webview-visual job confirms five failures across the Azure and Extra Body stories because the screenshots still use the old layout. Please regenerate the container-authored baselines from the uploaded webview-visual-regression artifact and commit them.
| "openAiStrictToolSchemas": "Strict tool schemas (strict: true)", | ||
| "openAiStrictToolSchemasDescription": "Send strict: true with tool schemas to enable strict schema adherence. Disable to use non-strict tool calling.", | ||
| "openAiUseResponsesApi": "Use Responses API", | ||
| "openAiUseResponsesApiDescription": "Send requests to the Responses API (/responses) instead of Chat Completions (/chat/completions). Enable this for endpoints that require the Responses API to combine reasoning (reasoning_effort) with tool use, such as Azure OpenAI deployments of GPT-6.1 Terra.", |
There was a problem hiding this comment.
These two keys were added only to the English locale. The current check-translations CI job fails because all 17 non-English webview locales are missing both keys. Please add the localized entries before merging.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Related GitHub Issue
No linked GitHub Issue. This PR fixes a bug reported in the Zoo Code Discord community.
Description
The bug. A user runs Zoo Code 3.98.0 with the OpenAI Compatible provider. The endpoint is Azure OpenAI behind a corporate API gateway. The Model ID is a custom Azure deployment name for GPT-6.1 Terra. Every request fails with:
Root cause. The OpenAI Compatible provider (
src/api/providers/openai.ts) only sends requests to Chat Completions. It sendsreasoning_efforttogether withtoolson that endpoint. GPT-6.1 Terra on Azure OpenAI rejects this combination on/chat/completions. The endpoint requires the Responses API for reasoning plus function tools. The user verified with curl that/v1/responsesworks on their backend. Turning reasoning off is not an option for them.The fix. This PR adds an opt-in Use Responses API setting (
openAiUseResponsesApi) to the OpenAI Compatible provider. When enabled,createMessageposts to{Base URL}/responsesinstead of/chat/completions:reasoning_effortis translated to the Responses API shapereasoning: { effort: ... }. Reasoning and tools are sent together. Neither is dropped.convertToResponsesApiInputandprocessResponsesApiStreamtransforms. The xAI provider already uses the same transforms.Chat Completions stays the default. Existing setups do not change.
Answers to the reporter's questions:
Note for reviewers: the Responses API path targets endpoints that implement the OpenAI Responses API. It is not meant for Azure AI Inference endpoints (
*.services.ai.azure.com), which use a different path shape.Test Procedure
src/api/providers/__tests__/openai.spec.tscover: routing to the Responses API when the setting is on,reasoning: { effort }translation with function tools present, reasoning omitted when disabled, stringtool_choicepassthrough, and Chat Completions staying the default when the setting is off.OpenAICompatible.spec.tsxcover the new checkbox: render, default-unchecked state, checked state, and save wiring.pnpm testsuite: 13,723 tests passed. Rancheck-typesforsrc,packages/types, andwebview-ui. Ran ESLint on the touched files with no new suppressions./responseswithreasoning.effortand tools together.Pre-Submission Checklist
src/test-utils/,@/utils/test-utils) are used only for mechanical duplication; test intent and scenario-specific mocks remain visible.*.visual.tsxsnapshot inwebview-ui/. Seewebview-ui/AGENTS.md→ "When a UI change needs a snapshot". — See "Visual Snapshots" below.Visual Snapshots
The change adds one checkbox to the OpenAI Compatible provider settings. The screenshot below shows the new Use Responses API checkbox (unchecked by default) with its description, rendered from the shipped
OpenAICompatiblecomponent in the repo's Playwright Story Gallery:Baseline regeneration needed. The new checkbox shifts the layout of the existing
openai-compatible-azureandopenai-compatible-extra-bodystories, so their committed baselines under__screenshots__/need an update. The authoring sandbox has no Docker and no Playwright browsers, so host-side regeneration was not possible. Perwebview-ui/AGENTS.md, the fallback is to dispatch.github/workflows/visual-regression.ymlagainst this branch and commit the container-generated baselines from thewebview-visual-regressionartifact.Videos (interaction / animation only)
Not applicable. The change adds a static checkbox with no new interaction flow.
Documentation Updates
Paired docs PR: see "Related PRs". It documents the Use Responses API option on the OpenAI Compatible provider page and adds a troubleshooting entry for the 400
reasoning_efforterror.Related PRs
Additional Notes
The reporter plans to roll this out to 200+ developers behind a corporate gateway and offered to test builds or share logs.
Get in Touch
Discord: taltas1