Skip to content

perf(pool): binary search for Cache waiters - #352

Merged
seanmonstar merged 1 commit into
hyperium:masterfrom
patrick-immometrica:perf-cache-waiter-lookup
Nov 16, 2026
Merged

seanmonstar merged 1 commit into
hyperium:masterfrom
patrick-immometrica:perf-cache-waiter-lookup

Conversation

@patrick-immometrica

Copy link
Copy Markdown
Contributor

store_waker runs on every poll of a CacheFuture and cancel_waiter on every drop, and both scanned the whole waiter queue under the lock. With many callers parked that gets quadratic: polling 30k waiters once took ~150ms.

Waiter ids only grow and are pushed to the back, so the queue is sorted and a binary search finds them. Same run now takes under 1ms. The id is u64 so it can't wrap on 32-bit targets.

store_waker runs on every poll of a CacheFuture and cancel_waiter on every
drop, and both scanned the whole waiter queue under the lock. With many
callers parked that gets quadratic: polling 30k waiters once took ~150ms.

Waiter ids only grow and are pushed to the back, so the queue is sorted
and a binary search finds them. Same run now takes under 1ms. The id is
u64 so it can't wrap on 32-bit targets.

@seanmonstar seanmonstar left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simple improvement, thanks!

@seanmonstar
seanmonstar merged commit 493b3db into hyperium:master Nov 16, 2026
20 checks passed
@patrick-immometrica
patrick-immometrica deleted the perf-cache-waiter-lookup branch November 18, 2026 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants