Repository navigation
fix(commons): close batch iterators and keep failure causes - #3456
Conversation
- BatchMapperIterator.close() closed only the origin iterator. A batch iterator returned by the mapper and left partly read, as happens when a traversal limit stops early, was never closed. close() now closes it first, as FlatMapperIterator already does. - MachineInfo.getIpAddress() cached an empty list before the lookup, so one failed lookup made every later call return [] without retrying. Cache the list only after the lookup succeeds, as getMacAddress() already does since apache#3335. - WrappedIterator.close(Iterator) and MachineInfo dropped the original exception when wrapping it. Pass it on as the cause.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #3456 +/- ##
============================================
+ Coverage 58.42% 58.43% +0.01%
+ Complexity 6903 3081 -3822
============================================
Files 779 779
Lines 69391 69391
Branches 9567 9567
============================================
+ Hits 40539 40550 +11
+ Misses 25179 25170 -9
+ Partials 3673 3671 -2
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
imbajin
left a comment
There was a problem hiding this comment.
Review summary
Three independent reviews found one resource-cleanup issue (8/10 each). The reviews did not independently run tests; a fix and regression test are in progress.
imbajin
left a comment
There was a problem hiding this comment.
+1. The fixes and final diff were independently re-reviewed; the reported issues are addressed and no new actionable issues were found (10/10). Targeted regression tests passed locally as noted in the discussion; Java 17 CI is still running.
Purpose of the PR
Found while scanning
hugegraph-commons/hugegraph-commonfor small, test-backed fixes. No issue was filed; each fix comes with a unit test that fails on master.BatchMapperIteratorhad noclose()override, soclose()closed only the origin iterator. A batch iterator returned by the mapper and left partly read was never closed. The Server hits this inGraphTransaction.queryAdjacentVerticesandHugeVertexStepByBatchwhen a traversal stops early, for example on a limit.FlatMapperIterator.close()already closes its batch iterator first.MachineInfo.getIpAddress()set its cache to an empty list before the lookup. If the lookup threw, every later call returned[]without retrying. PD'sLicenseVerifierServicereads this list. fix(commons): handle missing MAC addresses #3335 fixed the same problem ingetMacAddress().WrappedIterator.close(Iterator)and twoMachineInfolookups wrapped an exception without passing it as the cause.Main Changes
BatchMapperIterator.close(): close the current batch iterator, then callsuper.close(). This mirrorsFlatMapperIterator, including its behaviour when the batch close throws: the exception propagates and the origin is not closed on that call.MachineInfo.getIpAddress(): build the list locally and cache it only after the lookup succeeds. A lookup that keeps failing now throws on each call instead of returning a cached[].WrappedIterator.close(Iterator)and inMachineInfo. Messages and exception types are unchanged.BatchMapperIteratorTestandMachineInfoTest. Both classes are already inUnitTestSuite.Verifying these changes
testCloseClosesCurrentBatchIterator(batch iterator left open),testCloseWithFailingBatchKeepsCause(nothing thrown),testBatchCloseFailureKeepsCauseandtestGetMacByUnboundAddressKeepsCause(cause is null),testGetIpAddressRetriesAfterLookupFailure(second call returns the cached[]).1e8f6bd23:mvn -ntp -B test -pl hugegraph-commons/hugegraph-common -am -Dtest=UnitTestSuite -DskipCommonsTests=false -Dsurefire.failIfNoSpecifiedTests=false -Deditorconfig.skip=trueResult:
UnitTestSuiteran 397 tests, against 392 on master atcd7f1493b, so all 5 new tests ran and passed.Two existing tests fail on this host on master too, with the same assertion:
MachineInfoTest.testGetIpAddressList:54andtestGetLocalAllInetAddress:74. The host has interface addresses that the test's IP patterns do not match; this PR does not touch those patterns. In this runRestClientTest.testPostHttpsWithAllParamsandtestPostHttpsWithTokenAndAllParamsalso timed out connecting to an external HTTPS host. This PR does not touchRestClient, and both tests passed in theUnitTestSuiterun of another branch on the same host minutes later.-Deditorconfig.skip=trueis there because master fails the editorconfig check onhelm/hugegraph/docs/rag.md(missing final newline), a file this PR does not touch.Does this PR potentially affect the following parts?
Documentation Status
Doc - TODO: required documentation is pending; complete it before merging.Doc - Done: documentation is included here or linked below.Doc - No Need: no user-visible documentation is affected.