Skip to content

fix: encode datetime as ISO 8601 in JSON request bodies - #1203

Closed
RaphaelFakhri wants to merge 3 commits into
apify:masterfrom
RaphaelFakhri:fix/json-body-datetime-isoformat
Closed

RaphaelFakhri wants to merge 3 commits into
apify:masterfrom
RaphaelFakhri:fix/json-body-datetime-isoformat

Conversation

@RaphaelFakhri

@RaphaelFakhri RaphaelFakhri commented Nov 5, 2026 •

Copy link
Copy Markdown

Description

JSON request bodies and JSON key-value store records were serialized with json.dumps(..., default=str). A datetime was sent as 2026-01-01 00:00:00+00:00 instead of ISO 8601, and a set or any custom object was stored as its str() representation without an error.

This change adds one shared default hook, json_default in _utils/http.py, and uses it in both encoders:

  • datetime, date and time are encoded with isoformat().
  • UUID is encoded with str().
  • Pydantic URLs are encoded as strings. Request models dumped in Python mode keep AnyUrl objects, for example request_url in WebhookClient.update(), which previously relied on str().
  • Any other type raises TypeError with the type name. For set_record(), the message also suggests passing bytes or a string with a matching content type.
  • allow_nan=False is unchanged.
  • Request queue batches are unchanged. Request models are dumped with model_dump(mode='json', fallback=str) first, so an unsupported value in userData is still sent as its str().

The hook lives in _utils/http.py because _utils/encoding.py imports http_clients, so importing it from http_clients/_base.py would create a circular import.

Issues

Testing

  • New unit tests:
    • test_prepare_request_call_encodes_datetime_as_iso_8601
    • test_prepare_request_call_encodes_time_and_uuid_as_strings
    • test_prepare_request_call_rejects_non_json_value (set, object)
    • test_encode_key_value_store_record_value_encodes_datetime_as_iso_8601
    • test_encode_key_value_store_record_value_rejects_non_json_value
    • test_update_sends_request_url_as_string: guards the webhook URL path.
  • Without the source change, 5 of the new tests fail. With it, all 8 pass. The full unit suite passes (2573 tests).
  • ruff check, ruff format --check, ty check, and poe check-docstrings pass.

JSON request bodies, JSON key-value store records and request queue batches were serialized with `json.dumps(..., default=str)`. A `datetime` went out as `2026-01-01 00:00:00+00:00`, with a space instead of `T`, and a `set` or any custom object was stored as its `str()` representation without an error.

All three encoders now share one `default` hook. It encodes `datetime` and `date` with `isoformat()`, encodes pydantic URLs (left in place by Python-mode `model_dump()`, for example the webhook `request_url`) as strings, and raises `TypeError` for any other type. For `set_record()`, the error also suggests passing bytes or a string with a matching content type.

Closes apify#1172
@github-actions github-actions Bot added the t-tooling Issues with this label are in the ownership of the tooling team. label Nov 5, 2026
Comment thread src/apify_client/_utils/http.py Resolved
Comment thread src/apify_client/_utils/encoding.py Resolved
Comment thread tests/unit/test_http_clients.py Resolved
@vdusek

vdusek commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Issue #1172 is in the 4.0 milestone because this change is breaking: calls that work on 3.4.0 would raise TypeError after it, for example a set, Decimal, or custom object in run input or push_items. We can't ship it in 3.x, so I'm closing this PR.

✍️ Drafted by Claude Code

@vdusek vdusek closed this Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JSON request bodies serialize unknown types with str(), including datetime

2 participants