Skip to content

Add optional response batching to LookupResources - #176

Merged
ostafen merged 5 commits into
mainfrom
feat/lr-batched-lookupresources
Nov 12, 2026
Merged

ostafen merged 5 commits into
mainfrom
feat/lr-batched-lookupresources

Conversation

@ostafen

@ostafen ostafen commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds an opt-in mechanism for LookupResources to return multiple resources per stream message, instead of one message per resource, with a single resume cursor per batch.

Changes

  • LookupResourcesRequest.optional_response_batch_size (uint32, field 9): when non-zero, the server may return up to this many resources per response message, in LookupResourcesResponse.batched. Zero (the default) preserves the existing one-resource-per-message behavior. The value trades time-to-first-result against throughput (~1000 is a good default); requests above the server-side maximum are clamped, not rejected.
  • LookupResourcesResponse.batched (LookupResourcesBatch, field 6): populated instead of the singular fields when batching is requested. A response message uses either the singular fields or batched, never both.
  • New LookupResourcesBatch message: repeated LookupResourcesResult results plus a single after_cursor for the whole batch. One cursor per message (not per result) keeps responses compact — a stream is consumed message-by-message, so only one resume point per message is ever needed.
  • New LookupResourcesResult message: the per-resource fields (resource_object_id, permissionship, partial_caveat_info); it carries no cursor of its own, since the batch resumes as a unit.

Recommended over non-batched going forward

New and existing integrations should prefer batched LookupResources (set optional_response_batch_size) over the one-resource-per-message form.

For any lookup returning more than a handful of resources, batching is substantially more efficient: it collapses N stream messages into N/batch_size, and — because the resume cursor lives on the batch rather than on every result — it removes the dominant per-message payload (the cursor), rather than merely amortizing the gRPC framing. In practice this is a multi-x throughput win on high-cardinality lookups.

The non-batched form is retained for backward compatibility and remains the default so existing clients are unaffected, but it should be treated as the legacy path: prefer batched responses for all new work, and migrate throughput-sensitive callers.

Compatibility

Fully backward compatible and opt-in — no behavior change unless optional_response_batch_size is set. Existing clients continue to receive one resource per message via the singular fields.

Generated clients and server support follow in dependent PRs.

Adds an opt-in way for LookupResources to return multiple resources per
stream message instead of one message per resource:

- LookupResourcesRequest.optional_response_batch_size (uint32, <= 1000):
  when non-zero, the server may return up to this many resources per
  response message. Zero (the default) preserves the existing
  one-resource-per-message behavior.
- LookupResourcesResponse.batched_results (repeated LookupResourcesResult):
  populated instead of the singular fields when batching is requested. A
  response message uses either the singular fields or batched_results.
- New LookupResourcesResult message mirroring the per-resource fields
  (resource_object_id, permissionship, partial_caveat_info,
  after_result_cursor).

Fully backward compatible and opt-in; no behavior change unless the new
request field is set. Batching reduces per-message overhead for
high-cardinality lookups.
@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

The latest Buf updates on your PR. Results from workflow Lint / lint (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedNov 11, 2026, 11:21 AM

@ostafen
ostafen marked this pull request as ready for review November 11, 2026 10:31
Replace the repeated LookupResourcesResult batched_results field with a
LookupResourcesBatch wrapper that holds the results plus one resume cursor
for the whole batch, and drop the per-result cursor from
LookupResourcesResult. A stream is consumed message-by-message, so only one
resume point per message is ever needed; carrying a cursor per result made
the cursor token dominate the response payload.
optional_response_batch_size no longer validates lte:1000 (a uint32 is
already non-negative). The maximum is now a server-side configuration flag,
documented in the field: larger batches favor throughput, smaller batches
favor time-to-first-result, ~1000 is a good default, and over-large requests
are clamped to the server maximum rather than rejected.
vroldanbet
vroldanbet previously approved these changes Nov 11, 2026

@vroldanbet vroldanbet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@ostafen
ostafen merged commit be7db46 into main Nov 12, 2026
3 checks passed
@ostafen
ostafen deleted the feat/lr-batched-lookupresources branch November 12, 2026 17:08
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 12, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants