Skip to content

fix(commons): close batch iterators and keep failure causes - #3456

Merged
imbajin merged 3 commits into
apache:masterfrom
bitflicker64:scan/common-1
Nov 8, 2026
Merged

imbajin merged 3 commits into
apache:masterfrom
bitflicker64:scan/common-1

Conversation

@bitflicker64

Copy link
Copy Markdown
Contributor

Purpose of the PR

Found while scanning hugegraph-commons/hugegraph-common for small, test-backed fixes. No issue was filed; each fix comes with a unit test that fails on master.

  • BatchMapperIterator had no close() override, so close() closed only the origin iterator. A batch iterator returned by the mapper and left partly read was never closed. The Server hits this in GraphTransaction.queryAdjacentVertices and HugeVertexStepByBatch when a traversal stops early, for example on a limit. FlatMapperIterator.close() already closes its batch iterator first.
  • MachineInfo.getIpAddress() set its cache to an empty list before the lookup. If the lookup threw, every later call returned [] without retrying. PD's LicenseVerifierService reads this list. fix(commons): handle missing MAC addresses #3335 fixed the same problem in getMacAddress().
  • WrappedIterator.close(Iterator) and two MachineInfo lookups wrapped an exception without passing it as the cause.

Main Changes

  • BatchMapperIterator.close(): close the current batch iterator, then call super.close(). This mirrors FlatMapperIterator, including its behaviour when the batch close throws: the exception propagates and the origin is not closed on that call.
  • MachineInfo.getIpAddress(): build the list locally and cache it only after the lookup succeeds. A lookup that keeps failing now throws on each call instead of returning a cached [].
  • Pass the caught exception as the cause in WrappedIterator.close(Iterator) and in MachineInfo. Messages and exception types are unchanged.
  • Add 5 tests to BatchMapperIteratorTest and MachineInfoTest. Both classes are already in UnitTestSuite.

Verifying these changes

  • Trivial rework / code cleanup without any test coverage. (No Need)
  • Already covered by existing tests, such as (please modify tests here).
  • Need tests and can be verified as follows:
    • On master code all 5 new tests fail: testCloseClosesCurrentBatchIterator (batch iterator left open), testCloseWithFailingBatchKeepsCause (nothing thrown), testBatchCloseFailureKeepsCause and testGetMacByUnboundAddressKeepsCause (cause is null), testGetIpAddressRetriesAfterLookupFailure (second call returns the cached []).
    • Ran on Linux x86_64, OpenJDK 17.0.20.1, Maven 3.8.7, 2026-11-07, at 1e8f6bd23:
      mvn -ntp -B test -pl hugegraph-commons/hugegraph-common -am -Dtest=UnitTestSuite -DskipCommonsTests=false -Dsurefire.failIfNoSpecifiedTests=false -Deditorconfig.skip=true
      Result: UnitTestSuite ran 397 tests, against 392 on master at cd7f1493b, so all 5 new tests ran and passed.
      Two existing tests fail on this host on master too, with the same assertion: MachineInfoTest.testGetIpAddressList:54 and testGetLocalAllInetAddress:74. The host has interface addresses that the test's IP patterns do not match; this PR does not touch those patterns. In this run RestClientTest.testPostHttpsWithAllParams and testPostHttpsWithTokenAndAllParams also timed out connecting to an external HTTPS host. This PR does not touch RestClient, and both tests passed in the UnitTestSuite run of another branch on the same host minutes later.
    • -Deditorconfig.skip=true is there because master fails the editorconfig check on helm/hugegraph/docs/rag.md (missing final newline), a file this PR does not touch.

Does this PR potentially affect the following parts?

Documentation Status

  • Doc - TODO: required documentation is pending; complete it before merging.
  • Doc - Done: documentation is included here or linked below.
  • Doc - No Need: no user-visible documentation is affected.

- BatchMapperIterator.close() closed only the origin iterator. A batch
  iterator returned by the mapper and left partly read, as happens when
  a traversal limit stops early, was never closed. close() now closes
  it first, as FlatMapperIterator already does.
- MachineInfo.getIpAddress() cached an empty list before the lookup, so
  one failed lookup made every later call return [] without retrying.
  Cache the list only after the lookup succeeds, as getMacAddress()
  already does since apache#3335.
- WrappedIterator.close(Iterator) and MachineInfo dropped the original
  exception when wrapping it. Pass it on as the cause.
@codecov

codecov Bot commented Nov 7, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 58.43%. Comparing base (8e327b8) to head (232a4be).
⚠️ Report is 12 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3456      +/-   ##
============================================
+ Coverage     58.42%   58.43%   +0.01%     
+ Complexity     6903     3081    -3822     
============================================
  Files           779      779              
  Lines         69391    69391              
  Branches       9567     9567              
============================================
+ Hits          40539    40550      +11     
+ Misses        25179    25170       -9     
+ Partials       3673     3671       -2     
Flag Coverage Δ *Carryforward flag
commons 96.37% <ø> (ø)
hstore 36.10% <ø> (+0.02%) ⬆️
pd 56.83% <ø> (+0.09%) ⬆️
server_hbase 42.59% <ø> (+0.01%) ⬆️ Carriedforward from 1e8f6bd
server_memory 56.52% <ø> (ø)
server_rocksdb 43.10% <ø> (+0.06%) ⬆️
store 35.93% <ø> (-0.03%) ⬇️

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

Three independent reviews found one resource-cleanup issue (8/10 each). The reviews did not independently run tests; a fix and regression test are in progress.

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. The fixes and final diff were independently re-reviewed; the reported issues are addressed and no new actionable issues were found (10/10). Targeted regression tests passed locally as noted in the discussion; Java 17 CI is still running.

@imbajin
imbajin merged commit 34f1026 into apache:master Nov 8, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants