Skip to content

Fix cubic processing time in the legacy_attrs extension - #1721

Open
naaz234 wants to merge 1 commit into
Python-Markdown:masterfrom
naaz234:legacy-attrs-cubic-backtracking
Open

naaz234 wants to merge 1 commit into
Python-Markdown:masterfrom
naaz234:legacy-attrs-cubic-backtracking

Conversation

@naaz234

@naaz234 naaz234 commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

Description

LegacyAttrs.handleAttributes ran ATTR_RE.sub (\{@([^\}]*)=([^\}]*)}) over every text and tail. At a {@ with no closing }, the first group runs to the end of the text, backtracks to each =, and the second group runs to the end again from each one. Every {@ repeats this, so text made of n unclosed {@a= costs O(n³). With extensions=['legacy_attrs'] on CPython 3.11:

'{@a=' * 500   (2 KB):  0.45 s -> 0.0005 s
'{@a=' * 1000  (4 KB):  3.1 s  -> 0.001 s
'{@a=' * 2000  (8 KB):  24.6 s -> 0.002 s
'{@a=' * 25000 (100 KB):         0.011 s

Neither group can contain }, so a definition that starts at {@ ends at the next }. If the first {@ before a } does not match, no later {@ before that } can either, because its text is a suffix of the first one's. handleAttributes now splits the text on } and tries ATTR_RE.match once per piece, at its first {@. ATTR_RE is unchanged and still defines the syntax.

I compared the returned text and the (name, value) calls to el.set, in order, against the old ATTR_RE.sub on all 47 million strings up to length 9 over {, @, =, }, a, newline and space, and on 300,000 random strings built from fragments like {@id=, }} and {{@. They were identical in every case. A deliberately broken variant of the new code was caught by the same comparison. The new test takes 26 s on master and 0.3 s with the fix.

AI Assistance Disclosure

  • No AI tools were used in preparing this PR.

  • AI tools were used and I have disclosed which ones, and have fully reviewed and verified their output.

    Claude Code (Claude Opus 5.5) was used for everything ticked below.

    AI tools were use to assist with the following:

    • Code
    • Code Comments
    • Documentation
    • Changelog Entry
    • Commit Message
    • The Description Above
    • Other: the regression test, the timing script used to find the slow input, and the comparison against the old regex

Checklist

`LegacyAttrs.handleAttributes` ran `ATTR_RE.sub` over each text. At a
`{@` with no closing `}`, the first group runs to the end of the text,
backtracks to every `=`, and the second group runs to the end again
from each one, so text made of n unclosed `{@A=` costs O(n^3): 8 KB
takes about 25 seconds to convert.

Neither group can contain `}`, so a definition that starts at `{@` ends
at the next `}`, and if the first `{@` before a `}` does not match, no
later one before that `}` can. Split the text on `}` and try
`ATTR_RE.match` once per piece, at its first `{@`. The text returned
and the attributes set are unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant