Skip to content

Partial fix for #15174: ValueFlow speedups - #9004

Open
danmar wants to merge 9 commits into
cppcheck-opensource:mainfrom
cppchecksolutions:fix-15174-exact
Open

danmar wants to merge 9 commits into
cppcheck-opensource:mainfrom
cppchecksolutions:fix-15174-exact

Conversation

@danmar

@danmar danmar commented Nov 8, 2026

Copy link
Copy Markdown
Collaborator

Partial fix for #15174. These commits make ValueFlow faster without limiting the analysis: nothing is switched off or removed. This is a first step; making normal level give about the same warnings as exhaustive is left for later PRs.

  • Commits 1 and 2 fix two cases where lifetime values were created or forwarded where they don't belong: a global variable's lifetime was forwarded through the rest of the file, and arr[i] was treated as a decayed pointer. Both were quadratic.
  • Commits 3, 5 and 6 avoid repeated work (filter once, cache, reuse a search result). They give the same results.
  • Commits 4 and 7 skip a function in the next ValueFlow iteration when none of its values changed, compared by a hash of the values.

Commits

  1. valueFlowForwardLifetime: do not forward lifetime of non-local variable past end of function (colobot restext.cpp: ValueFlow 615 s → 5 s)
  2. valueFlowLifetime: subscripted array is not a decayed pointer (spice rop3.c: ValueFlow 352 s → 28 s)
  3. isAliasOf: filter the alias values once per reference
  4. ValueFlow: skip functions without new values in the next iteration
  5. SymbolDatabase: cache getEnumType() during setValueTypeInTokenList()
  6. Tokenizer: avoid quadratic backward search for commas in setVarIdParseDeclaration()
  7. ValueFlow: compare a hash of the values when skipping unchanged functions

Results on 10,019 Debian packages

This branch vs main, both with --check-level=exhaustive. The packages are random Debian source packages with 3–800 C/C++ files and at most 40 MB. Each package was checked on its own (-j1) with a 20-minute timeout. Times are whole Cppcheck CPU time.

Speed (9,853 packages that both versions finish)

main CPU per package Packages main this PR Saving
under 1 s 3,779 1,260 s 1,110 s −12%
1–10 s 3,360 12,860 s 10,382 s −19%
10–100 s 2,017 68,209 s 51,423 s −25%
100 s – 20 min 697 230,263 s 158,253 s −31%
all 9,853 87 h 61 h −29%
  • ValueFlow time alone: −33%.
  • Median per-package CPU time is 81% of main (packages ≥ 1 s).
  • 33 packages time out with main and finish with this PR.
  • At normal level, a smaller run on 1,308 of the packages took about 23% less time.

Warnings

Of 4,962,744 warnings, 11 are lost and 55 are new. Every difference was reproduced and traced to a commit:

Change Count Commit Correct?
lost danglingLifetime, objectIndex 11 1 false positives removed
new knownConditionTrueFalse 16 4 correct
new nullPointerOutOfMemory / nullPointerOutOfResources 9 2 correct
new uninitvar (simh), arrayIndexOutOfBoundsCond (funtools) 2 2 false positives
new, in mailfromd 28 – noise: output differs between runs also with main

Why commit 4 gives more warnings: in main, running the passes again on an unchanged function can add a possible value to a comparison, and that keeps a known value from being set later. All 16 new warnings have this shape:

for (i = 0; i < 3; i++) {
    if (i == 0) ...
    else if (i == 1) ...
    else if (i == 2) ...   // Condition 'i==2' is always true
}

testrunner: 5,392 tests, 0 failures.

🤖 Generated with Claude Code

dmcppcheck and others added 9 commits November 8, 2026 10:21
…le past end of function

For a global variable the scope body end is null so the lifetime was
forwarded through the rest of the file, into unrelated functions.

colobot restext.cpp: ValueFlow 615s => 5s.
In 10000 Debian packages (--check-level=exhaustive) this removes 11
false positives (danglingLifetime, objectIndex) and adds no warnings.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
For 'arr[i]' where arr is an array of pointers, isDecayedPointer()
returned true because the type of 'arr[i]' is a pointer. A lifetime
value was then created and forwarded for each subscript; with many
'arr[N] = ...' statements in one function this is quadratic.

spice rop3.c: ValueFlow 352s => 28s.
In 10000 Debian packages (--check-level=exhaustive) this adds 11
warnings: 9 correct nullPointerOutOfMemory/nullPointerOutOfResources
and 2 false positives (uninitvar, arrayIndexOutOfBoundsCond).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The values of the reference token were filtered again for every node
in the expression. Filter them once and skip the AST search when there
is nothing that can match.

No changed warnings in a 50 package debian corpus, ValueFlow ~10% faster.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The iterated ValueFlow passes are run again for all functions until no
new values are found. A function where no value changed in the previous
iteration will not get new values from the intra-function passes, so
skip it.

Running the passes again on an unchanged function can add a possible
value to a comparison, which then keeps a known value from being set
later. In 10000 Debian packages (--check-level=exhaustive) this adds 16
correct knownConditionTrueFalse warnings and loses none.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
getEnumType() scans the whole enum body each time a token with enum type
gets a valuetype. With big enums that are used a lot this is quadratic.
Cache the result per enum scope while the valuetypes are set; the result
is only cached when every literal it looked at already had a valuetype.

crispy-doom: total time 124s => 78s. No changed warnings.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…eDeclaration()

For a name after a comma the tokens before it were searched back to
';', '(', '[' or '{'. In a long comma separated list of names this search
was repeated from every element. Reuse the result of the search from the
previous comma.

crispy-doom: total time 78s => 64s. No changed warnings.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…ions

Skipping a function in the next iteration when its number of values did
not change missed changes where a value is replaced or changes kind
(possible -> known) without changing the count. For example a call whose
return value got known in valueFlowFunctionReturn. Compare a hash of all
values in the function instead.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Only some passes skip the unchanged functions. Another pass can remove a
value that a skipping pass then adds again, so the function's values look
unchanged after the iteration. When such a function was skipped only the
removal happened, and the values kept changing in the next iterations.
In selfcheck lib/mathlib.cpp then needed 6 iterations instead of 4 and
gave 'ValueFlow maximum iterations exceeded'.

Now a function is skipped only if its values are unchanged and the
skipping passes did not change it in the previous iteration.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@danmar danmar added the ai-assisted contains code which was written with the assistance of AI - please review extra carefully label Nov 8, 2026

@dmcppcheck dmcppcheck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review

I found no problems that I could reproduce, and no warning changed compared with base on the inputs I tried. Testing found no false positives, false negatives or crashes from the lifetime changes, the alias filter, the enum-type cache or the comma-search cache. Two remarks from reading the code, both about the new "skip unchanged functions" logic:

  • hashValue() leaves out some Value fields, at least indirect and container. If a value changes only in one of those fields, the function's hash stays the same and the function can be skipped by mistake.
  • Whether a pass "uses skippedFunctions" is decided by looking for the text skippedFunctions in the pass name, which is the stringified macro argument. This is fragile.

No earlier review comments exist.

Testing

  • Built cppcheck and testrunner in pr/, and cppcheck in base/.
  • ./testrunner (all tests): 5392 tests, 0 failures. The new tests in TestValueFlow pass.
  • Lifetime checks, compared with base. All gave the same results as base:
    • A global pointer or member pointer set to a local inside a nested block and used after the block in the same function. danglingLifetime and invalidLifetime are still reported.
    • The same inside a lambda inside a member function.
    • A global std::vector<int*> given arr[0] = &x through push_back, and returning arr[0] that holds &x. Both are still detected.
    • 2-D arrays int a[3][4]; int *p = a[1]; return p;. Base does not warn either, so this is not a regression.
  • Corpus comparison, base vs PR, on test/cfg/.c and samples/ with --check-level=exhaustive --enable=warning,style,performance,portability: the output is identical.
  • cppcheck's own sources, base vs PR, on lib/{mathlib,token,checkother,astutils,library,checkstl,programmemory,checkbufferoverrun}.cpp with --debug-warnings --library=cppcheck-lib:
    • Non-bailout output is identical, apart from line shifts in astutils.cpp, which the PR edits.
    • No "ValueFlow maximum iterations exceeded" message appeared.
    • Wall time went from 404 s (base) to 202 s (PR).
  • I did not build a test input that gets skipped wrongly because of the hash fields it leaves out. That remark comes from reading the code.

Automated review of 4cc4ad2 by Claude. The pull request was built and tested locally.

Comment thread lib/valueflow.cpp
combine(static_cast<std::size_t>(v.conditional));
combine(static_cast<std::size_t>(v.moveKind));
combine(static_cast<std::size_t>(v.lifetimeKind));
combine(static_cast<std::size_t>(v.lifetimeScope));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review

The hash leaves out indirect, container and a few other fields of ValueFlow::Value. If a pass replaces a value with one that differs only in indirect (for example indirect 0 vs 1 for uninit or null values) or only in container, the function's hash stays the same. The function is then skipped in the next iteration even though its values changed. Consider adding at least indirect and container to the hash.

Comment thread lib/valueflow.cpp
F mRun;
ValueFlowPassAdaptor(const char* pname, bool pcpp, F prun) : ValueFlowPass(), mName(pname), mCPP(pcpp), mRun(prun) {}
ValueFlowPassAdaptor(const char* pname, bool pcpp, F prun)
: ValueFlowPass(), mName(pname), mCPP(pcpp), mUsesSkippedFunctions(std::strstr(pname, "skippedFunctions") != nullptr), mRun(prun) {}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review

Whether a pass honours skippedFunctions is found by searching for the text "skippedFunctions" in its stringified name. A pass that is renamed, or that gets the set through a helper without that literal text in the VFA(...) argument, is quietly misclassified. Then changedBySkippingPass is not tracked for it, which brings back the iteration problem that commit 4cc4ad2 fixed. An explicit flag in the VFA macro, such as a VFA_SKIP variant, would be more robust.

@danmar

danmar commented Nov 8, 2026

Copy link
Copy Markdown
Collaborator Author

@pfultz2 I would be interested in your opinion about these tweaks. This is made by AI, however I feel it's pretty well tested and has very promising results. Some tweaks were needed to make CI happy so I am extending the testing, the scripts are measuring performance and warnings on more debian packages with the latest changes..

If your on-going work in valueflow/symboldatabase/etc will be messed up then feel free to say that we should delay this until your changes are in..

@danmar

danmar commented Nov 8, 2026

Copy link
Copy Markdown
Collaborator Author

I could split this up into several PRs also if wanted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-assisted contains code which was written with the assistance of AI - please review extra carefully

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants