Skip to content

fix(openai): serialize tool results that hold datetimes, bytes or models - #8086

Open
gjija wants to merge 1 commit into
google:mainfrom
gjija:fix-openai-tool-result-json
Open

gjija wants to merge 1 commit into
google:mainfrom
gjija:fix-openai-tool-result-json

Conversation

@gjija

@gjija gjija commented Nov 10, 2026

Copy link
Copy Markdown

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

2. Or, if no issue exists, describe the change:

Problem:

OpenAILlm builds each tool message with a bare json.dumps(part.function_response.response) (_openai_llm.py:147). Tool results reach the adapter as Python objects: _caller.py wraps a non-dict result as {'result': value} and keeps it as is. So when a tool returns a datetime, bytes or a Pydantic model, the next request raises TypeError and the run stops.

ADK's own load_memory tool returns a Pydantic model (LoadMemoryResponse), so every OpenAILlm agent that uses load_memory fails on the turn after the tool runs. The same agent works on Gemini.

Repro (a real InMemoryRunner, load_memory_tool and OpenAILlm; only chat.completions.create is stubbed):

import asyncio

from google.adk.agents import LlmAgent
from google.adk.events.event import Event
from google.adk.integrations.openai import OpenAILlm
from google.adk.runners import InMemoryRunner
from google.adk.tools.load_memory_tool import load_memory_tool
from google.genai import types
from openai import AsyncOpenAI
from openai.types.chat import ChatCompletion

calls = []


async def create(**kwargs):
  calls.append(kwargs["messages"])
  if len(calls) == 1:
    call = {"id": "c1", "type": "function",
            "function": {"name": "load_memory", "arguments": '{"query": "cat"}'}}
    message = {"role": "assistant", "content": None, "tool_calls": [call]}
    reason = "tool_calls"
  else:
    message = {"role": "assistant", "content": "Your cat is Tom."}
    reason = "stop"
  return ChatCompletion.model_validate({
      "id": "x", "object": "chat.completion", "created": 0, "model": "gpt-4o",
      "choices": [{"index": 0, "finish_reason": reason, "message": message}],
  })


async def main():
  client = AsyncOpenAI(api_key="unused")
  client.chat.completions.create = create
  agent = LlmAgent(
      name="a",
      model=OpenAILlm(model="gpt-4o", client=client),
      tools=[load_memory_tool],
  )
  runner = InMemoryRunner(agent=agent)
  old = await runner.session_service.create_session(
      app_name=runner.app_name, user_id="u"
  )
  old.events.append(
      Event(author="user", content=types.UserContent("My cat is Tom."))
  )
  await runner.memory_service.add_session_to_memory(old)
  session = await runner.session_service.create_session(
      app_name=runner.app_name, user_id="u"
  )
  async for event in runner.run_async(
      user_id="u",
      session_id=session.id,
      new_message=types.UserContent("What is my cat called?"),
  ):
    if event.content and event.content.parts[0].text:
      print("reply:", event.content.parts[0].text)
  print("tool message:", [m for m in calls[-1] if m["role"] == "tool"][0]["content"])


asyncio.run(main())

On main:

  File ".../integrations/openai/_openai_llm.py", line 147, in _content_to_openai_messages
    json.dumps(part.function_response.response)
TypeError: Object of type LoadMemoryResponse is not JSON serializable

A tool that returns {"created": datetime(...)} fails the same way: TypeError: Object of type datetime is not JSON serializable.

Solution:

Pass a default= hook to that json.dumps. The hook calls pydantic_core.to_jsonable_python with the same settings as the Gemma fix (#7668): by_alias=False, exclude_none=True, bytes_mode="base64", serialize_unknown=True. So datetimes become ISO 8601 text, bytes become base64 text, models become dicts of their non-None fields, and values Pydantic cannot convert fall back to their str().

json.dumps only calls default for values it cannot encode itself, so every tool result that works today is sent byte for byte the same (same ensure_ascii, same layout). Only results that used to raise change. The sibling OpenAIResponsesLlm in the same package already passes default=str (_openai_responses_llm.py:169-170).

With this change the repro prints:

reply: Your cat is Tom.
tool message: {"result": {"memories": [{"content": {"parts": [{"text": "My cat is Tom."}], "role": "user"}, "custom_metadata": {}, "id": "...", "author": "user", "timestamp": "..."}]}}

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

New parametrized test test_content_to_openai_messages_encodes_tool_result_values in tests/unittests/integrations/openai/test_openai_llm.py, with one case each for a datetime, bytes, a Pydantic model, a LoadMemoryResponse, and a plain JSON result (non-ASCII text and a nested None) whose output must stay exactly what json.dumps gives today.

Without the source change, the four non-JSON cases fail with their own TypeError and the plain case passes (4 failed, 66 passed). With it:

$ pytest tests/unittests/integrations/openai/test_openai_llm.py
70 passed
$ pytest tests/unittests/integrations/openai
296 passed

pyink and isort leave both files unchanged.

Manual End-to-End (E2E) Tests:

I don't have an OpenAI key, so the repro above stands in for a live run: a real InMemoryRunner, InMemoryMemoryService, load_memory_tool and OpenAILlm, with chat.completions.create stubbed to return real ChatCompletion objects. Its output on main and with this change is shown above.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

Additional context

Values json.dumps rejects without calling default (dict keys that are not str, int, float, bool or None, and circular references) still raise as before; that is out of scope here.

OpenAILlm sent a tool result with a bare json.dumps. A tool that returns
a datetime, bytes or a Pydantic model made the next request raise
TypeError, so the run failed on the first tool turn. ADK's own
load_memory tool returns such a model, so every OpenAILlm agent that
uses it failed. The same result sent to Gemini works.

json.dumps now gets a default that converts these values with
pydantic_core.to_jsonable_python, as the Gemma fix does: ISO datetimes,
base64 bytes, models as dicts with None fields dropped and field names
not aliased, and str() as a last resort. The default only runs on
values that raised before, so results that already worked are sent
unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants