[danmar]

Replace the discontinued PCRE library with PCRE2 for rules

PCRE (8.x) is end of life and is being removed from Linux distributions.

• lib/regex.cpp: use the PCRE2 API (with JIT when available)
• fix endless loop when a rule pattern matches an empty string
• CMake, Makefile, dmake and Visual Studio project use PCRE2
• CI installs/builds PCRE2 instead of PCRE
• update documentation

Rule files and the "pcre" engine name are unchanged.

[danmar]

@fliiiix @andrew-aitchison @loqs @Harvester57 @pfultz2 @firewave
this PR replaces PCRE with PCRE2.
So it's more aggressive than #8382 .
I have implemented this with AI so any help with reviewing / testing it out would be appreciated.

[dmcppcheck]

AI review

I found no new problems. lib/regex.cpp now uses the PCRE2 API correctly. The match data is created on each call, so a Regex that several threads share is safe. A failed JIT compile falls back to the interpreter. The match loop steps forward one byte after an empty match, so patterns such as b* or ( x)? no longer loop forever. Error messages for patterns that do not compile are the same as before (\ at end of pattern, missing terminating ] for character class), so the existing expected strings in the tests still pass. Patterns that match an empty string now report a found '' finding at many positions. Cppcheck's de-duplication cuts that down to one per line, and the same behaviour lets zero-width rules such as (?= x) work, so I don't count it as a defect. I could not check the Windows/MSVC and CI changes (library names, install prefix) here.

Testing

Build: The container has the libpcre2-8 runtime (10.42, libpcre2-8.so.0) but no headers and no pcre2-config. To build anyway, I wrote a small pcre2.h that declares only the functions regex.cpp uses (they map to the _8 symbols), plus a fake pcre2-config (--cflags/--libs8). With those I built make -j8 HAVE_RULES=yes cppcheck testrunner in pr/, and it built without errors. Base was not built: PCRE1 is not available, so I could not compare against the old engine.

Unit tests: ./testrunner TestRegExPcre TestCmdlineParser TestCppcheck ran 406 tests with 0 failures, including the new emptymatch and the compile-error message checks in TestCmdlineParser.

Python tests: pytest rules_test.py gave 6 passed. pytest other_test.py helloworld_test.py inline-suppress-polyspace_test.py -k rule gave 6 passed.

Manual checks with cppcheck --rule=... on a small C file:

• ( x)? (matches the empty string) finishes at once. It reports the empty matches, which are de-duplicated to one per line, as well as the real x matches.
• (?= x) (zero-width lookahead) reports the right lines.
• \bx\b gives correct results, so word boundaries work when matching starts partway into the string.
• Running on 30 files with -j8 and an empty-matching rule finishes with stable output and no hang.
• (?<=x gives failed to compile rule pattern '(?<=x' (missing closing parenthesis).
• A pattern that recurses without end, (a|aa)*+(?:(?R)), gives the error JIT stack limit reached (pos: 0) [pcre_exec] instead of crashing or hanging.

---

Automated review of ab7764c by Claude. The pull request was built and tested locally.

[Harvester57]

Hi there,

Built and tested locally by cherry-picking the PR over v2.22.0, applies cleanly and works perfectly well, thank you very much !

[dmcppcheck]

AI review

I found one new problem, in the match loop in lib/regex.cpp. When an empty match is found after the start offset (ovector[0] == ovector[1] > pos), the loop moves pos to that position. The next pcre2_match() call then finds the same empty match again, so matchFn is called twice for it. An example is a lookahead such as (?=b) on "aab", which reports (2,2) twice.

In the cppcheck CLI the duplicate is hidden, because identical error messages are de-duplicated. Even so, Regex::match() returns the wrong result. Comparing against the match start fixes it: pos = (pos2 > pos1) ? pos2 : pos2 + 1;. The existing emptymatch test does not cover this, because there the empty match is at the start offset.

I found no other problems. The rest of the PCRE2 port, the build changes and the docs look consistent, and no PCRE1 references are left apart from the pcre_exec error id, which is kept on purpose.

Earlier comments

#	Comment	By	Where	Status
1	requirements.txt could use PCRE2Project/[email protected] instead of a full URL	@pfultz2	requirements.txt:1	✅ Fixed

Testing

Build: The container has only the libpcre2-8.so.0 runtime, with no headers and no pcre2-config. I wrote a minimal pcre2.h that maps the used functions to their _8 symbols, plus a fake pcre2-config. With those, make -j8 HAVE_RULES=yes cppcheck testrunner in pr/ built without errors. I did not build base/, because PCRE1 is not available here. I could not test the Windows, MSVC and CI changes.

Unit tests: ./testrunner TestRegExPcre TestCmdlineParser TestCppcheck gave Tests failed: 0.

Duplicate empty match. I linked a small driver against the PR's lib/regex.o:

#include <string>
#include "regex.h"
#include <cstdio>
int main(int, char**argv){ std::string err; auto r=Regex::create(argv[1],Regex::Engine::Pcre,err);
 if(!r){puts(err.c_str());return 1;}
 err = r->match(argv[2],[](int a,int b){printf("(%d,%d)\n",a,b);}); puts(err.c_str());}

$ g++ -std=c++11 -DHAVE_RULES -Ilib t.cpp lib/regex.o -lpcre2-8 -o t
$ ./t '(?=b)' aab
(2,2)
(2,2)
$ ./t 'x*' aaxx
(0,0)
(1,1)
(2,4)

The match at (2,2) is reported twice. A standalone program that runs the same loop directly on libpcre2 shows the same thing for (?= x) on " a x b x": (2,2), (2,2), (6,6), (6,6).

In the CLI, cppcheck -q --rule='(?= x)' a.c prints one found '' per line, because cppcheck de-duplicates identical messages. So the problem shows only at the Regex API level.

---

Automated review of e209e12 by Claude. The pull request was built and tested locally.

[dmcppcheck]

AI review

An empty match found after pos (pos1 == pos2 > pos) is reported twice. pos is set to pos2, the next pcre2_match() finds the same empty match at that offset, and only then does the loop step forward. For example, (?=b) on "aab" calls matchFn(2,2) twice. The CLI's message de-duplication hides this, but Regex::match() itself reports the match twice.

Compare against the match start instead:

pos = (pos2 > pos1) ? pos2 : (pos2 + 1);

It would also help to add a test like emptymatch that uses a lookahead such as (?=b) on "aab" and expects exactly one match.