Skip to content

[Fix] Reasoning with tool use fails on OpenAI Compatible endpoints that require the Responses API - #2498

Draft
zoomote2[bot] wants to merge 1 commit into
mainfrom
fix/openai-compatible-responses-api-1ybm193i5siuq
Draft

zoomote2[bot] wants to merge 1 commit into
mainfrom
fix/openai-compatible-responses-api-1ybm193i5siuq

Conversation

@zoomote2

@zoomote2 zoomote2 Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

​Opened on behalf of @taltas. Follow up by mentioning @roomote, in the web UI, or in Discord.

Related GitHub Issue

No linked GitHub Issue. This PR fixes a bug reported in the Zoo Code Discord community.

Description

The bug. A user runs Zoo Code 3.98.0 with the OpenAI Compatible provider. The endpoint is Azure OpenAI behind a corporate API gateway. The Model ID is a custom Azure deployment name for GPT-6.1 Terra. Every request fails with:

400 Function tools with reasoning_effort are not supported for <deployment> in /v1/chat/completions.
To use function tools, use /v1/responses or set reasoning_effort to 'none'.

Root cause. The OpenAI Compatible provider (src/api/providers/openai.ts) only sends requests to Chat Completions. It sends reasoning_effort together with tools on that endpoint. GPT-6.1 Terra on Azure OpenAI rejects this combination on /chat/completions. The endpoint requires the Responses API for reasoning plus function tools. The user verified with curl that /v1/responses works on their backend. Turning reasoning off is not an option for them.

The fix. This PR adds an opt-in Use Responses API setting (openAiUseResponsesApi) to the OpenAI Compatible provider. When enabled, createMessage posts to {Base URL}/responses instead of /chat/completions:

  • The provider keeps its custom Base URL, custom Model ID (including Azure deployment names), custom headers, and custom model info.
  • The resolved reasoning_effort is translated to the Responses API shape reasoning: { effort: ... }. Reasoning and tools are sent together. Neither is dropped.
  • Tools use the flattened Responses function shape. The existing strict-tool-schemas setting and the MCP strict exemption still apply.
  • Message conversion and stream parsing reuse the shared convertToResponsesApiInput and processResponsesApiStream transforms. The xAI provider already uses the same transforms.
  • Temperature and max-output-token handling mirror the existing Chat Completions policies.

Chat Completions stays the default. Existing setups do not change.

Answers to the reporter's questions:

  1. Is GPT-6.1 Terra support in 3.98.0 native-provider only? Yes. The 3.98.0 release added GPT-6.1 Terra to the native OpenAI provider model list (and to the OpenAI Codex and VS Code LM lists). The native OpenAI provider uses the Responses API for all models, so reasoning plus tools work there. Nothing changed for the OpenAI Compatible provider in 3.98.0.
  2. Can OpenAI Compatible use the Responses API today? No. Before this PR there was no setting or flag. The provider always used Chat Completions.
  3. Can the native OpenAI provider use a custom base URL plus a custom deployment name? Partly. It supports a custom base URL. It does not support custom model IDs or Azure deployment names. Unknown IDs fall back to the default model. So it cannot serve this setup.
  4. Can Responses API support be added to OpenAI Compatible? Yes. That is this PR.

Note for reviewers: the Responses API path targets endpoints that implement the OpenAI Responses API. It is not meant for Azure AI Inference endpoints (*.services.ai.azure.com), which use a different path shape.

Test Procedure

  • New unit tests in src/api/providers/__tests__/openai.spec.ts cover: routing to the Responses API when the setting is on, reasoning: { effort } translation with function tools present, reasoning omitted when disabled, string tool_choice passthrough, and Chat Completions staying the default when the setting is off.
  • New webview tests in OpenAICompatible.spec.tsx cover the new checkbox: render, default-unchecked state, checked state, and save wiring.
  • Ran the full pnpm test suite: 13,723 tests passed. Ran check-types for src, packages/types, and webview-ui. Ran ESLint on the touched files with no new suppressions.
  • Manual check for reviewers: pick the OpenAI Compatible provider, enable Use Responses API, point the Base URL at a Responses-capable endpoint, and run a task with a reasoning model. Requests should go to /responses with reasoning.effort and tools together.

Pre-Submission Checklist

  • Issue Linked: This PR is linked to an approved GitHub Issue (see "Related GitHub Issue" above). — No issue exists; the report came from Discord.
  • Scope: My changes are focused on the linked issue (one major feature/fix per PR).
  • Self-Review: I reviewed my own code with care.
  • Writing Style: The PR title and description use Simplified Technical English (ASD-STE100): short sentences, plain words, active voice, one idea per sentence, no contractions.
  • Testing: New and/or updated tests have been added to cover my changes (if applicable).
  • Test Helpers (test changes only): Shared test helpers (src/test-utils/, @/utils/test-utils) are used only for mechanical duplication; test intent and scenario-specific mocks remain visible.
  • Visual Snapshot (UI changes only): If a user would notice this change at a glance (layout, theme tokens, brand elements, empty/error states), I added or updated a *.visual.tsx snapshot in webview-ui/. See webview-ui/AGENTS.md → "When a UI change needs a snapshot". — See "Visual Snapshots" below.
  • Documentation Impact: I checked if my changes need documentation updates (see "Documentation Updates" section below).
  • Contribution Guidelines: I read and agree to the Contributor Guidelines.

Visual Snapshots

The change adds one checkbox to the OpenAI Compatible provider settings. The screenshot below shows the new Use Responses API checkbox (unchecked by default) with its description, rendered from the shipped OpenAICompatible component in the repo's Playwright Story Gallery:

New "Use Responses API" checkbox in the OpenAI Compatible provider settings

Baseline regeneration needed. The new checkbox shifts the layout of the existing openai-compatible-azure and openai-compatible-extra-body stories, so their committed baselines under __screenshots__/ need an update. The authoring sandbox has no Docker and no Playwright browsers, so host-side regeneration was not possible. Per webview-ui/AGENTS.md, the fallback is to dispatch .github/workflows/visual-regression.yml against this branch and commit the container-generated baselines from the webview-visual-regression artifact.

Videos (interaction / animation only)

Not applicable. The change adds a static checkbox with no new interaction flow.

Documentation Updates

  • No documentation updates are required.
  • Yes, documentation updates are required. (Describe what to update, or link to a PR in the docs repository).

Paired docs PR: see "Related PRs". It documents the Use Responses API option on the OpenAI Compatible provider page and adds a troubleshooting entry for the 400 reasoning_effort error.

Related PRs

Additional Notes

The reporter plans to roll this out to 200+ developers behind a corporate gateway and offered to test builds or share logs.

Get in Touch

Discord: taltas1

…ible provider

Reasoning models such as GPT-6.1 Terra on Azure OpenAI reject
reasoning_effort combined with function tools on /chat/completions and
require /v1/responses. The OpenAI Compatible provider only spoke Chat
Completions, so reasoning could not be combined with tool use on those
endpoints.

Add an openAiUseResponsesApi setting. When enabled, createMessage posts
to {baseUrl}/responses with the provider's custom base URL, custom
model/deployment ID, and custom headers. The resolved reasoning_effort
is translated to the Responses reasoning.effort shape, tools use the
flattened Responses function shape, and the shared Responses input and
stream transforms (also used by the xAI provider) handle conversion.
Chat Completions stays the default, so existing setups are unchanged.
@coderabbitai

coderabbitai Bot commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one): 1

  • coderabbit-review-active

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@zoomote2

zoomote2 Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor Author

5 issues outstanding. See session

  • src/api/providers/openai.ts:344 — Responses API ignores the disabled-streaming setting.
  • src/api/providers/openai.ts:405 — Responses API drops configured Extra Body fields.
  • src/api/providers/openai.ts:342 — Responses API does not sanitize lone surrogates in prompts and history.
  • webview-ui/src/i18n/locales/en/settings.json:602-603 — New setting strings are missing from all non-English locales; check-translations fails.
  • webview-ui/src/components/settings/providers/OpenAICompatible.tsx:203-212 — Visual baselines were not regenerated; webview-visual reports five failures.

Reviewed 05e80b1

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Review status

This PR was opened by an automated account. A human maintainer must verify the change intent, provenance, and validation before merging.

Current step: Fix the failing required CI checks; awaiting-maintainer requires CI and automated review completion.

Review-state labels are managed by this workflow; do not edit them manually. community-approved is managed the same way — do not add or remove it manually. It signals a fresh community code approval for the current head as an advisory priority only; maintainer review is still required.

model: modelId,
instructions: systemPrompt,
input: convertToResponsesApiInput(messages),
stream: true,

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When openAiUseResponsesApi is combined with openAiStreamingEnabled: false, this still sends stream: true, so the existing Enable streaming setting is silently ignored. Please honor the non-streaming configuration on this path (and cover that combination), or prevent the unsupported combination explicitly.

try {
let stream: AsyncIterable<OpenAI.Responses.ResponseStreamEvent>
try {
stream = await this.client.responses.create(

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This request never passes through withExtraBody, unlike every Chat Completions request in this provider. Enabling Responses API therefore silently drops configured Extra Body fields such as service_tier or gateway metadata. Merge this.extraBody into the Responses request with the existing request-owned-field precedence and add a regression test.


const requestBody: OpenAI.Responses.ResponseCreateParamsStreaming = {
model: modelId,
instructions: systemPrompt,

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing OpenAI-compatible paths sanitize lone UTF-16 surrogates in the system prompt and converted message history, but this path forwards both unchanged (convertToResponsesApiInput does not sanitize them). A task containing a sliced surrogate can therefore be rejected again when Responses API is enabled. Please preserve the existing sanitization invariant for this request body and add a Responses-path test.

{t("settings:providers.openAiStrictToolSchemasDescription")}
</div>
</div>
<div>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This visible layout change needs updated committed Playwright baselines under webview-ui/AGENTS.md. The current webview-visual job confirms five failures across the Azure and Extra Body stories because the screenshots still use the old layout. Please regenerate the container-authored baselines from the uploaded webview-visual-regression artifact and commit them.

"openAiStrictToolSchemas": "Strict tool schemas (strict: true)",
"openAiStrictToolSchemasDescription": "Send strict: true with tool schemas to enable strict schema adherence. Disable to use non-strict tool calling.",
"openAiUseResponsesApi": "Use Responses API",
"openAiUseResponsesApiDescription": "Send requests to the Responses API (/responses) instead of Chat Completions (/chat/completions). Enable this for endpoints that require the Responses API to combine reasoning (reasoning_effort) with tool use, such as Azure OpenAI deployments of GPT-6.1 Terra.",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two keys were added only to the English locale. The current check-translations CI job fails because all 17 non-English webview locales are missing both keys. Please add the localized entries before merging.

@codecov

codecov Bot commented Nov 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 61.36364% with 17 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/api/providers/openai.ts 60.46% 12 Missing and 5 partials ⚠️

📢 Thoughts on this report? Let us know!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant