Skip to content

Avoid rescanning delimiter stacks for the rule of 3 - #1720

Open
naaz234 wants to merge 1 commit into
Python-Markdown:masterfrom
naaz234:emphasis-rule-of-3-rescan
Open

naaz234 wants to merge 1 commit into
Python-Markdown:masterfrom
naaz234:emphasis-rule-of-3-rescan

Conversation

@naaz234

@naaz234 naaz234 commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

Description

>>> markdown.markdown('*a **b*a **b' * 20000)   # 234 KiB, no extensions, CPython 3.11
master: 27.4 s    with this change: 1.6 s
(58 KiB: 0.75 s -> 0.22 s, 117 KiB: 4.0 s -> 0.46 s)

When the rule of 3 (#1691) keeps an ambiguous closer from pairing with the top of its delimiter stack, DelimiterProcessor._handle_match walks the stack down for an opener of the same run length that the rule allows. In the input above each ** opener never closes and stays on the stack, so every * closer for which that search fails walks all of them again, which is O(n^2).

Stack entries are only appended as new tuples or popped, so while the entry at the depth a failed search reached is still in place, nothing below it has changed and the next search for that token and length can stop there. Output is identical to master on 300,760 renders (all strings up to length 8 over *, _, a and space, plus 63,000 random and repeated-pattern documents, with no extensions and with extra, smarty and legacy_em). The skip is taken 3,011 times in that set, and a planted off-by-one in it changes the output. The added test takes 4.2 s on master and 0.6 s with the change.

AI Assistance Disclosure

  • No AI tools were used in preparing this PR.

  • AI tools were used and I have disclosed which ones, and have fully reviewed and verified their output.

    AI tools were use to assist with the following:

    • Code
    • Code Comments
    • Documentation
    • Changelog Entry
    • Commit Message
    • The Description Above
    • Other: finding the slow input, the regression test and the output comparison against master

Checklist

When the rule of 3 keeps an ambiguous closer from pairing with the top
of its delimiter stack, the stack is walked down for an opener of the
same run length that the rule allows. Unclosed openers that can never
pair stay on the stack, so each closer for which that search fails walks
all of them again, and `*a **b` repeated n times takes O(n^2) time.

Stack entries are only ever appended as new tuples or popped, so if the
entry at the depth a failed search reached is still in place, nothing
below it has changed. The processor records that depth and entry per
token and run length, and a later search for the same length stops
there.
@facelessuser

Copy link
Copy Markdown
Collaborator

It doesn't seem that this introduces any new cases of dropped text or notable slowdowns in other areas, and it does seem to improve speed in the targeted case.

I would say this change looks good to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants