Skip to content

[Pipe] Fix aggregate window statistics - #19326

Open
Caideyipi wants to merge 2 commits into
masterfrom
fix/pipe-aggregate-window-statistics
Open

Caideyipi wants to merge 2 commits into
masterfrom
fix/pipe-aggregate-window-statistics

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Summary

  • Fix standard aggregate statistics for negative values, integer extremes, and overflowing intermediate sums.
  • Preserve legacy window-state restoration semantics and reject collisions between explicit and default output names.

Example

A tumbling window over INT64 values [-4, -9] emits peak=9 with the INT64 schema. A peak from Long.MIN_VALUE saturates at Long.MAX_VALUE instead of wrapping.

Verification

  • mvn clean test -pl iotdb-core/datanode -am -T 1 -Dtest=AggregateProcessorTest,TumblingWindowingProcessorTest -Dsurefire.failIfNoSpecifiedTests=false (26 tests passed)
  • mvn spotless:apply -P with-integration-tests -pl iotdb-core/datanode -am (passed)
  • git diff --cached --check (passed)

@Caideyipi
Caideyipi marked this pull request as draft November 20, 2026 03:08
@Caideyipi

Copy link
Copy Markdown
Collaborator Author

Self-review summary for base 52b42d310b3758734f8f131c8cfea55fedd42808, reviewed head 260a0088c9bb4396d642f00246d276d385befccd, and final head 1b9ea5af45e01aa167cb055a1d880da44320a25b.

  • Finding: TSDataTypeSwitchArchitectureTest identified type switches outside TypeService in PeakOperator, AbsoluteMaxOperator, and IntegralPoweredSumOperator.
  • Fix: Route those dispatches through TsFile's TypeService, preserving the typed reads, numeric conversions, and serialized-state handling.
  • Verification: DataNode Spotless passed. A clean local DataNode reactor run passed all 27 selected tests: 1 architecture test, 22 aggregate processor tests, and 4 tumbling-window tests.
  • GitHub Actions for the final commit were queued or running at the time of verification; remote CI remains pending.

@Caideyipi
Caideyipi marked this pull request as ready for review November 20, 2026 04:12
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@codecov

codecov Bot commented Nov 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 67.03297% with 90 lines in your changes missing coverage. Please review.
✅ Project coverage is 49.48%. Comparing base (52b42d3) to head (1b9ea5a).
⚠️ Report is 5 commits behind head on master.

Files with missing lines Patch % Lines
...pe/processor/aggregate/TimeSeriesRuntimeState.java 25.00% 18 Missing ⚠️
...teresult/sametype/numeric/AbsoluteMaxOperator.java 67.34% 16 Missing ⚠️
...b/pipe/processor/aggregate/AggregateProcessor.java 75.43% 14 Missing ⚠️
...t/sametype/numeric/IntegralPoweredSumOperator.java 46.15% 14 Missing ⚠️
...fictype/doubletype/FractionPoweredSumOperator.java 46.66% 8 Missing ⚠️
...gregate/window/datastructure/TimeSeriesWindow.java 86.36% 6 Missing ⚠️
...regatedresult/standardstatistics/PeakOperator.java 72.72% 3 Missing ⚠️
...tedresult/standardstatistics/VarianceOperator.java 0.00% 2 Missing ⚠️
...result/specifictype/integertype/CountOperator.java 71.42% 2 Missing ⚠️
...processor/StandardStatisticsOperatorProcessor.java 0.00% 2 Missing ⚠️
... and 5 more
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #19326      +/-   ##
============================================
+ Coverage     49.30%   49.48%   +0.18%     
  Complexity      717      717              
============================================
  Files          5648     5654       +6     
  Lines        417020   418009     +989     
  Branches      55108    55253     +145     
============================================
+ Hits         205608   206870    +1262     
+ Misses       211412   211139     -273     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants