Skip to content

perf(client): make pool waiter cleanup amortized O(1) on Checkout drop - #345

Open
Nacl-HCl wants to merge 2 commits into
hyperium:masterfrom
Nacl-HCl:fix-pool-waiter-cleanup-4245
Open

Nacl-HCl wants to merge 2 commits into
hyperium:masterfrom
Nacl-HCl:fix-pool-waiter-cleanup-4245

Conversation

@Nacl-HCl

@Nacl-HCl Nacl-HCl commented Nov 2, 2026 •

Copy link
Copy Markdown

Problem

Fixes hyperium/hyper#4245

In the legacy client connection pool, every dropped Checkout with a parked waiter triggers clean_waiters, which runs VecDeque::retain over the entire waiter queue for that key while holding the pool lock. With n parked waiters this is O(n^2) total work — the reporter measured 10.5s for 131,072 concurrent requests against a cold pool.

Fix

put already skips canceled waiters lazily (tx.is_canceled() + pop_front), so the eager full scan on drop is redundant. clean_waiters now only pops the canceled prefix of the queue: each parked sender is popped at most once, making cleanup amortized O(1) per drop. The key entry is still removed once the queue is empty, so the observable cleanup contract is unchanged (test_pool_checkout_drop_cleans_up_waiters passes unmodified).

Verification

  • cargo test --all-features --lib: 107 passed, 0 failed (includes 2 new regression tests: bulk drop cleanup, and out-of-order drop followed by put still waking the live waiter).
  • cargo fmt --all --check: clean.
  • Timing harness (park 20k checkouts, drop all): 15.67s before → 22.6ms after.
  • The description, docs, and comments (words meant for humans) are written by a human, not by an LLM.

@Nacl-HCl
Nacl-HCl force-pushed the fix-pool-waiter-cleanup-4245 branch 2 times, most recently from d033a81 to 046ae75 Compare November 15, 2026 08:04
@Nacl-HCl

Copy link
Copy Markdown
Author

Note on the latest CI run (43692818514, attempt 2): it reports failure only because two jobs — "Check MSRV (1.85) on macos-latest" and "Test stable on macos-latest" — were cancelled before running any step (queued 15:47 UTC, cancelled 16:02 UTC, no steps executed). The other 16 checks on this head passed, including the other macOS jobs ("Test beta on macos-latest" and "Test nightly on macos-latest"), so this looks like a runner-side cancellation rather than a failure from this change. Re-running the two cancelled jobs should clear it.

@seanmonstar

Copy link
Copy Markdown
Member

Reducing the cost of repeat scans is a good goal! Though, I noticed that this would change it so that some canceled waiters will be left around in the list until something else, like putting a new value, happens. So it could result in more more kept around for longer.

The lazy cleanup in clean_waiters only pops the canceled prefix, so a
canceled waiter parked behind a long-lived live waiter stays in the
queue with no bound on how many can pile up.

Bound it: clean_waiters now counts drops that leave a canceled sender
behind a live waiter, and once that count reaches half the queue it
does a full retain and resets the count. Stale senders can no longer
accumulate without bound, and the rescan stays amortized O(1) since it
only runs after at least len/2 such drops.

Adds regression test test_pool_checkout_drop_bounds_stale_waiters,
which fails on the previous version.
@Nacl-HCl
Nacl-HCl force-pushed the fix-pool-waiter-cleanup-4245 branch from 046ae75 to f53f9b0 Compare November 18, 2026 07:47
@Nacl-HCl

Copy link
Copy Markdown
Author

Good catch — you're right, the lazy version can leave canceled senders parked behind a long-lived live waiter with no bound on how many pile up. I've updated the PR to bound that: clean_waiters now counts drops that leave a canceled sender behind a live waiter, and once that count reaches half the queue length it does a full retain and resets the count. So stale senders can't accumulate without bound, and the rescan stays amortized O(1) because it only runs after at least len / 2 such drops. Added test_pool_checkout_drop_bounds_stale_waiters covering it (it fails on the previous version).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

hyper-util legacy client: each dropped pool Checkout scans every parked waiter under the pool lock, O(n²) at cold start

2 participants