Repository navigation
gh-161420: Process mimalloc's delayed frees on every generic malloc again - #161533
Conversation
…lloc again Since the update to mimalloc 2.5.2 (pythongh-159111), _mi_malloc_generic() processes the heap's delayed frees only every 100 calls. A block over MI_MEDIUM_OBJ_SIZE_MAX has a page of its own, so a free of it from another thread is a delayed free, and every allocation of such a block is a generic malloc: each thread could keep up to about 100 large blocks that other threads had freed. Process the delayed frees on every generic malloc again, as mimalloc 2.1 did; the deferred-free callback and the periodic collection stay batched.
corona10
added
needs backport to 3.14
corona10
left a comment
corona10
left a comment
There was a problem hiding this comment.
Please write descption to be concisely as possible.
|
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
Co-authored-by: Donghee Na <[email protected]>
|
All commit authors signed the Contributor License Agreement. |
|
I have made the requested changes; please review again |
|
Thanks for making the requested changes! @corona10: please review the changes made to this pull request. |
|
Thanks @maikschulze for the PR, and @corona10 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14, 3.15. |
|
GH-161537 is a backport of this pull request to the 3.15 branch. |
|
GH-161538 is a backport of this pull request to the 3.14 branch. |
Since mimalloc 2.5.2 (gh-159111),
_mi_malloc_generic()processes delayed frees from other threads only every 100th call, so blocks over 64 KiB that other threads freed can pile up, up to about 100 per heap. This processes them on every generic malloc again, as before.Peak RSS of the example from the issue, free-threaded release builds:
mainNo measurable difference for small objects (pyperf, macOS and Linux).
I used an AI assistant to investigate this and draft the patch; I reviewed and tested the changes myself.