Repository navigation
Regular expression removal - #32
Closed
JaredDavis22 wants to merge 17 commits into
Closed
JaredDavis22 wants to merge 17 commits into
JaredDavis22 wants to merge 17 commits into
Conversation
…point to match java numeric parsing.
Contributor
Author
|
I believe it is ready for review now. I used some prior code in Literals to fail numeric detection faster. jmh-resultNRE5.json
...
|
Contributor
Author
|
Another cleanup pending. Encapsulation of ParseNumberResults was horrible. |
Change 4 leading spaces to a tab
Contributor
Author
|
fasterParsington fork has this in main. |
ctrueden
added a commit
that referenced
this pull request
Nov 18, 2026
Numeric literals of every kind (decimal, octal, hex and binary) are now
parsed by one hand-written scanner in parseNumber, rather than by four
regular expressions tried in turn. The 0x/0b prefix selects the radix,
and the rest of the grammar (fractional part, exponent, type suffix)
differs only slightly per radix. Integer digits are accumulated directly
rather than via Integer/Long.parseInt, so no NumberFormatException is
thrown and caught for long or BigInteger values.
The parseHex, parseBinary, parseOctal and parseDecimal methods are
removed; parseNumber is now the single entry point for numeric literals.
Because this is a breaking API change, the major version digit is now 4.
There are two intentional behavior changes, both aligning more closely
with Java:
1. A leading zero denotes octal only for integer literals. Previously,
e.g. 07.5 parsed as octal 7 followed by .5; now it parses as the
double 7.5.
2. A decimal literal may begin with a decimal point, e.g. .5 or -.5e2.
Previously, these were errors ("Invalid character"). Literals are
only parsed where an operand is expected, so a decimal point after an
operand is still the dot operator (e.g. a.b, x.' or a.*b).
JMH results (mvn -Pbenchmark, -f 2 -wi 4 -w 1 -i 5 -r 1),
in average time per operation, before -> after:
LiteralsBenchmark.parseSingleLiteral (ns/op)
INT 119.5 -> 26.3 -78%
DOUBLE 169.0 -> 21.1 -88%
SCIENTIFIC 143.2 -> 20.1 -86%
LONG 146.0 -> 33.3 -77%
FLOAT 158.9 -> 17.2 -89%
HEX 73.1 -> 31.1 -57%
BINARY 85.5 -> 25.1 -71%
BIG_INTEGER 1539.7 -> 119.8 -92%
STRING 27.8 -> 29.3 ~
BOOLEAN 1.5 -> 1.4 ~
ParsingBenchmark (us/op) parsePostfix parseTree
CONSTANT 0.151 -> 0.009 -94% 0.154 -> 0.010 -94%
VARIABLE 0.020 -> 0.021 ~ 0.023 -> 0.023 ~
ARITHMETIC 1.827 -> 0.392 -79% 1.963 -> 0.544 -72%
QUADRATIC 1.087 -> 0.638 -41% 1.243 -> 0.746 -40%
NESTED_GROUPS 0.790 -> 0.471 -40% 0.835 -> 0.571 -32%
FUNCTIONS 0.980 -> 0.347 -65% 1.076 -> 0.395 -63%
LONG_SUM 4.638 -> 4.451 -4% 5.244 -> 5.314 ~
OPERATORS_COMMON 1.540 -> 0.818 -47% 1.697 -> 0.796 -53%
OPERATORS_RARE 0.428 -> 0.457 +7% 0.514 -> 0.491 -4%
INT_LIST 1.984 -> 0.203 -90% 2.020 -> 0.207 -90%
DOUBLE_LIST 2.164 -> 0.326 -85% 2.194 -> 0.335 -85%
MIXED_LIST 3.170 -> 0.376 -88% 3.281 -> 0.418 -87%
Overall: -69% (geomean). For comparison, the regex-free rewrite in
PR #32 measured -56% overall on the same machine and settings.
OPERATORS_RARE and LONG_SUM contain no numeric literals, so their
changes reflect run-to-run variation.
Closes #24 and #32.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Member
|
With 2bb7700, I am hopeful that this issue is now sufficiently addressed. That commit implements the same idea conceptually (hand-written parsing rather than regexes), but does so more succinctly, with a net gain of 7 LoC, rather than 606 lines as in this PR, while also benchmarking faster than this PR on my machine. It also fixes a couple of non-Java-aligned behavior in the numeric parsing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Replace regular expressions with hard coded methods.
Performance improves for most operations.
There is probably a way to reduce the LONG_SUM benchmark impact. I will look at it later this week.
Improved
###Declined
Other benchmarks are within 5% and not show.
Json benchmark results attached. Upload them to https://jmh.morethan.io/ to see full performance results.
jmh-resultNRE2.json
jmh-resultMain.json