Skip to content

Regular expression removal - #32

Closed
JaredDavis22 wants to merge 17 commits into
scijava:mainfrom
JaredDavis22:regularExpressionRemoval
Closed

JaredDavis22 wants to merge 17 commits into
scijava:mainfrom
JaredDavis22:regularExpressionRemoval

Conversation

@JaredDavis22

Copy link
Copy Markdown
Contributor

Replace regular expressions with hard coded methods.

Performance improves for most operations.

There is probably a way to reduce the LONG_SUM benchmark impact. I will look at it later this week.

Improved

# Benchmark Params Mode/Unit Score Error
0 LiteralsBenchmark - parseSingleLiteral  633% token=SCIENTIFIC avgt in ns/op 53573 27310
1 LiteralsBenchmark - parseSingleLiteral  609% token=INT avgt in ns/op 31244 588
2 LiteralsBenchmark - parseSingleLiteral  502% token=BINARY avgt in ns/op 27145 607
3 ParsingBenchmark - parseTree  500% expression=DOUBLE_LIST avgt in us/op 61 0.7910.111
4 ParsingBenchmark - parsePostfix  480% expression=CONSTANT avgt in us/op 0.3440.059 0.0110.011
5 LiteralsBenchmark - parseSingleLiteral  465% token=LONG avgt in ns/op 46382 5315
6 LiteralsBenchmark - parseSingleLiteral  444% token=DOUBLE avgt in ns/op 35966 5913
7 ParsingBenchmark - parseTree  441% expression=CONSTANT avgt in us/op 0.3490.065 0.0880.007
8 LiteralsBenchmark - parseSingleLiteral  425% token=FLOAT avgt in ns/op 42581 27515
9 ParsingBenchmark - parseTree  402% expression=INT_LIST avgt in us/op 50.995 40.128
10 ParsingBenchmark - parsePostfix  400% expression=DOUBLE_LIST avgt in us/op 51 0.6280.448
11 ParsingBenchmark - parsePostfix  351% expression=INT_LIST avgt in us/op 40.888 0.8080.079
12 ParsingBenchmark - parsePostfix  310% expression=ARITHMETIC avgt in us/op 40.976 0.5360.163
13 ParsingBenchmark - parseTree  300% expression=ARITHMETIC avgt in us/op 41 0.7930.216
14 EvaluationBenchmark - evaluate  246.64193881297584% expression=BOOLEAN:mode=PARSE_AND_EVAL:strategy=STACK avgt in us/op 3.0140.87 0.3250.132
15 EvaluationBenchmark - evaluate  242.52962773545002% expression=BOOLEAN:mode=PARSE_AND_EVAL:strategy=TREE avgt in us/op 3.0560.892 0.5920.155
16 EvaluationBenchmark - evaluate  241.1556105466983% expression=ARITH_INT:mode=PARSE_AND_EVAL:strategy=STACK avgt in us/op 3.4661.016 0.0450.02
17 EvaluationBenchmark - evaluate  226.86897406409355% expression=POWER:mode=PARSE_AND_EVAL:strategy=STACK avgt in us/op 1.4370.44 0.0820.091
18 LiteralsBenchmark - parseSingleLiteral  224% token=HEX avgt in ns/op 21767 4021
19 EvaluationBenchmark - evaluate  208.2168142179887% expression=POWER:mode=PARSE_AND_EVAL:strategy=TREE avgt in us/op 1.4640.475 0.0190.013
20 EvaluationBenchmark - evaluate  206.68241851613152% expression=ARITH_DOUBLE:mode=PARSE_AND_EVAL:strategy=TREE avgt in us/op 3.3211.083 0.1410.028
21 EvaluationBenchmark - evaluate  205.67381746465955% expression=ARITH_DOUBLE:mode=PARSE_AND_EVAL:strategy=STACK avgt in us/op 3.0661.003 0.0920.095
22 EvaluationBenchmark - evaluate  200.14378673255084% expression=ARITH_INT:mode=PARSE_AND_EVAL:strategy=TREE avgt in us/op 3.6081.202 0.2850.247
23 ParsingBenchmark - parseTree  200% expression=OPERATORS_COMMON avgt in us/op 62 40.093
24 ParsingBenchmark - parseTree  200% expression=MIXED_LIST avgt in us/op 93 30.152
25 ParsingBenchmark - parsePostfix  169% expression=NESTED_GROUPS avgt in us/op 20.743 0.0390.145
26 ParsingBenchmark - parsePostfix  167% expression=MIXED_LIST avgt in us/op 83 20.134
27 ParsingBenchmark - parsePostfix  143% expression=FUNCTIONS avgt in us/op 20.823 0.6230.159
28 ParsingBenchmark - parseTree  106% expression=NESTED_GROUPS avgt in us/op 20.973 0.2670.01
29 LiteralsBenchmark - parseSingleLiteral  103% token=BIG_INTEGER avgt in ns/op 3,5831,763 208212
30 ParsingBenchmark - parsePostfix  100% expression=OPERATORS_COMMON avgt in us/op 42 0.6350.279
31 ParsingBenchmark - parseTree  100% expression=FUNCTIONS avgt in us/op 21 0.3930.163
32 EvaluationBenchmark - evaluate  89.93156239065134% expression=MIXED:mode=PARSE_AND_EVAL:strategy=STACK avgt in us/op 2.2381.178 0.3720.179
33 EvaluationBenchmark - evaluate  86.05098003821571% expression=MIXED:mode=PARSE_AND_EVAL:strategy=TREE avgt in us/op 2.3331.254 0.4430.224
34 LiteralsBenchmark - parseSingleLiteral  50% token=BOOLEAN avgt in ns/op 32 0.2970.213
35 ParsingBenchmark - parsePostfix  50% expression=QUADRATIC avgt in us/op 32 0.5190.25
36 ParsingBenchmark - parseTree  50% expression=QUADRATIC avgt in us/op 32 0.5030.339

###Declined

# Benchmark Params Mode/Unit Score Error
0 ParsingBenchmark - parsePostfix  -29% expression=LONG_SUM avgt in us/op 57 0.6853
1 ParsingBenchmark - parsePostfix  -5% expression=VARIABLE avgt in us/op 0.040.042 0.010.002
2 ParsingBenchmark - parseTree  -5% expression=VARIABLE avgt in us/op 0.0480.05 0.010.009

Other benchmarks are within 5% and not show.

Json benchmark results attached. Upload them to https://jmh.morethan.io/ to see full performance results.

jmh-resultNRE2.json
jmh-resultMain.json

@JaredDavis22

Copy link
Copy Markdown
Contributor Author

I believe it is ready for review now.

I used some prior code in Literals to fail numeric detection faster.

jmh-resultNRE5.json
jmh-resultMain.json

image

...

image

@JaredDavis22

Copy link
Copy Markdown
Contributor Author

Another cleanup pending. Encapsulation of ParseNumberResults was horrible.

@JaredDavis22

Copy link
Copy Markdown
Contributor Author

fasterParsington fork has this in main.

https://github.com/JaredDavis22/fasterParsington

ctrueden added a commit that referenced this pull request Nov 18, 2026
Numeric literals of every kind (decimal, octal, hex and binary) are now
parsed by one hand-written scanner in parseNumber, rather than by four
regular expressions tried in turn. The 0x/0b prefix selects the radix,
and the rest of the grammar (fractional part, exponent, type suffix)
differs only slightly per radix. Integer digits are accumulated directly
rather than via Integer/Long.parseInt, so no NumberFormatException is
thrown and caught for long or BigInteger values.

The parseHex, parseBinary, parseOctal and parseDecimal methods are
removed; parseNumber is now the single entry point for numeric literals.
Because this is a breaking API change, the major version digit is now 4.

There are two intentional behavior changes, both aligning more closely
with Java:

1. A leading zero denotes octal only for integer literals. Previously,
   e.g. 07.5 parsed as octal 7 followed by .5; now it parses as the
   double 7.5.

2. A decimal literal may begin with a decimal point, e.g. .5 or -.5e2.
   Previously, these were errors ("Invalid character"). Literals are
   only parsed where an operand is expected, so a decimal point after an
   operand is still the dot operator (e.g. a.b, x.' or a.*b).

JMH results (mvn -Pbenchmark, -f 2 -wi 4 -w 1 -i 5 -r 1),
in average time per operation, before -> after:

  LiteralsBenchmark.parseSingleLiteral (ns/op)
    INT                119.5 ->  26.3   -78%
    DOUBLE             169.0 ->  21.1   -88%
    SCIENTIFIC         143.2 ->  20.1   -86%
    LONG               146.0 ->  33.3   -77%
    FLOAT              158.9 ->  17.2   -89%
    HEX                 73.1 ->  31.1   -57%
    BINARY              85.5 ->  25.1   -71%
    BIG_INTEGER       1539.7 -> 119.8   -92%
    STRING              27.8 ->  29.3   ~
    BOOLEAN              1.5 ->   1.4   ~

  ParsingBenchmark (us/op)      parsePostfix           parseTree
    CONSTANT          0.151 -> 0.009 -94%   0.154 -> 0.010 -94%
    VARIABLE          0.020 -> 0.021 ~      0.023 -> 0.023 ~
    ARITHMETIC        1.827 -> 0.392 -79%   1.963 -> 0.544 -72%
    QUADRATIC         1.087 -> 0.638 -41%   1.243 -> 0.746 -40%
    NESTED_GROUPS     0.790 -> 0.471 -40%   0.835 -> 0.571 -32%
    FUNCTIONS         0.980 -> 0.347 -65%   1.076 -> 0.395 -63%
    LONG_SUM          4.638 -> 4.451  -4%   5.244 -> 5.314 ~
    OPERATORS_COMMON  1.540 -> 0.818 -47%   1.697 -> 0.796 -53%
    OPERATORS_RARE    0.428 -> 0.457  +7%   0.514 -> 0.491  -4%
    INT_LIST          1.984 -> 0.203 -90%   2.020 -> 0.207 -90%
    DOUBLE_LIST       2.164 -> 0.326 -85%   2.194 -> 0.335 -85%
    MIXED_LIST        3.170 -> 0.376 -88%   3.281 -> 0.418 -87%

Overall: -69% (geomean). For comparison, the regex-free rewrite in
PR #32 measured -56% overall on the same machine and settings.
OPERATORS_RARE and LONG_SUM contain no numeric literals, so their
changes reflect run-to-run variation.

Closes #24 and #32.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@ctrueden

Copy link
Copy Markdown
Member

With 2bb7700, I am hopeful that this issue is now sufficiently addressed. That commit implements the same idea conceptually (hand-written parsing rather than regexes), but does so more succinctly, with a net gain of 7 LoC, rather than 606 lines as in this PR, while also benchmarking faster than this PR on my machine. It also fixes a couple of non-Java-aligned behavior in the numeric parsing.

@ctrueden ctrueden closed this Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants