Skip to content

Fail the build when storm-client or storm-server imports relocated packages directly - #9286

Open
dragonknightk wants to merge 2 commits into
apache:masterfrom
dragonknightk:check-unshaded-imports
Open

dragonknightk wants to merge 2 commits into
apache:masterfrom
dragonknightk:check-unshaded-imports

Conversation

@dragonknightk

@dragonknightk dragonknightk commented Nov 18, 2026 •

Copy link
Copy Markdown

Fixes #7641

What is the purpose of the change

storm-client and storm-server now fail validate when their main code imports one of the relocated packages of storm-shaded-deps. Such an import still compiles if another dependency brings in the original jar, so the module silently depends on an unshaded copy. Six imports already did this:

  • storm-client Utils used org.apache.commons.io.IOUtils, which reaches storm-client through commons-compress.
  • storm-server EquivalenceUtils, SchedulerConfigCache, WorkerTokenManager, CgroupManager and Slot used com.google.common, which reaches storm-server only through the optional auto-service dependency.

They now use the shaded classes, like the other imports in the same modules.

The root pom defines the check-unshaded-imports checkstyle execution once, in pluginManagement, with an IllegalImport rule that reads the unshaded.illegalPkgs property. The shared storm_checkstyle.xml stays unchanged. The execution is bound to the phase in the unshaded.check.phase property, which defaults to none. storm-client and storm-server set it to validate and set their own unshaded.illegalPkgs. A comment next to the relocations in storm-shaded-deps/pom.xml points to these two properties.

The two lists differ:

  • storm-client bans all relocated packages.
  • commons-io, commons-lang3, commons-collections4, json-smart and snakeyaml are direct dependencies of storm-server, so their relocated packages are allowed there.
  • storm-server bans com.google.common and com.google.thirdparty, the com.google packages in the shaded jar, so com.google.auto.service stays allowed.

Both lists ban org.apache.curator, so new Curator code in the main code of storm-client and storm-server has to use the shaded package. This fits the ongoing Curator decoupling.

Other modules are not checked, because storm-core, storm-webapp and the external modules declare some of the relocated packages themselves. They inherit the execution with phase none, so it never runs there. Test code is not checked, because tests use curator-test, which is not shaded. The check covers import statements only, not fully qualified class names in code.

How was the change tested

Without the fixes, storm-client reports the IOUtils import and storm-server reports the five com.google.common imports, and nothing else. A temporary class in storm-server that imports one class from each relocated package fails on the 15 banned packages and passes on the allowed ones and on auto-service. Temporary imports of the first, the last and two other banned packages fail validate in storm-client and in storm-server. The effective pom of storm-client and storm-server binds check-unshaded-imports to validate with their own package lists. The build with the examples and externals profiles passes validate in every module and runs check-unshaded-imports only in storm-client and storm-server. UtilsTest passes in storm-client. SlotTest, WorkerTokenTest, EquivalenceUtilsTest, TestCgroups, TestResourceAwareScheduler and MultitenantSchedulerTest pass in storm-server.

AI Disclosure

Human-led contribution. Grok Bot, working for dragonknightk, assisted with this change, including the code, tests, and description. All submitted content was reviewed by a human before submission.

(。•̀ᴗ-)✧

…ckages directly

Closes issue 7641

## What is the purpose of the change

storm-client and storm-server now fail validate when their main code imports one of the relocated packages of storm-shaded-deps. Such an import still compiles if another dependency brings in the original jar, so the module silently depends on an unshaded copy. Six imports already did this:
- storm-client Utils used org.apache.commons.io.IOUtils, which reaches storm-client through commons-compress.
- storm-server EquivalenceUtils, SchedulerConfigCache, WorkerTokenManager, CgroupManager and Slot used com.google.common, which reaches storm-server only through the optional auto-service dependency.

They now use the shaded classes, like the other imports in the same modules.

Each module gets its own checkstyle execution with an IllegalImport rule, so the shared storm_checkstyle.xml stays unchanged for other modules. A comment next to the relocations in storm-shaded-deps/pom.xml asks to update both lists together.

The two lists differ:
- storm-client bans all relocated packages.
- commons-io, commons-lang3, commons-collections4, json-smart and snakeyaml are direct dependencies of storm-server, so their relocated packages are allowed there.
- storm-server bans com.google.common and com.google.thirdparty, the com.google packages in the shaded jar, so com.google.auto.service stays allowed.

Other modules are not checked, because storm-core, storm-webapp and the external modules declare some of the relocated packages themselves. Test code is not checked, because tests use curator-test, which is not shaded. The check covers import statements only, not fully qualified class names in code.

## How was the change tested

Without the fixes, storm-client reports the IOUtils import and storm-server reports the five com.google.common imports, and nothing else. A temporary class in storm-server that imports one class from each relocated package fails on the 15 banned packages and passes on the allowed ones and on auto-service. The build with the examples and externals profiles passes validate in every module. UtilsTest passes in storm-client. SlotTest, WorkerTokenTest, EquivalenceUtilsTest, TestCgroups, TestResourceAwareScheduler and MultitenantSchedulerTest pass in storm-server.

## AI Disclosure

Human-led contribution. Grok Bot, working for dragonknightk, assisted with this change, including the code, tests, and description. All submitted content was reviewed by a human before submission.

Signed-off-by: dragonknightk <[email protected]>
@rzo1
rzo1 requested review from GGraziadei and reiabreu November 18, 2026 08:47
@rzo1 rzo1 added this to the 3.3.0 milestone Nov 18, 2026

@rzo1 rzo1 left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thx for the contrib. The client list matches the relocations in storm-shaded-deps, and the storm-server exceptions are justified by its direct dependencies. One optional nit inline.

Comment thread storm-client/pom.xml Outdated
<execution>
<!-- Fail the build if main code imports one of the relocated packages of storm-shaded-deps.
Keep the list in sync with the relocated packages in storm-shaded-deps/pom.xml. -->
<id>check-unshaded-imports</id>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this execution is copied almost verbatim into storm-server; only illegalPkgs differs. You could define the check-unshaded-imports execution once in the root pom's checkstyle pluginManagement, reading illegalPkgs from a property that each module sets. Then the comment in storm-shaded-deps only has to point at one place. Fine as a follow-up.

Comment thread storm-server/pom.xml Outdated
<module name="Checker">
<module name="TreeWalker">
<module name="IllegalImport">
<property name="illegalPkgs" value="org.apache.thrift, com.google.common,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth noting in the description: this also bans direct org.apache.curator imports in main code. That fits the ongoing Curator decoupling, since any new Curator use has to go through the shaded package.

Closes issue 7641

## What is the purpose of the change

storm-client and storm-server now fail validate when their main code imports one of the relocated packages of storm-shaded-deps. Such an import still compiles if another dependency brings in the original jar, so the module silently depends on an unshaded copy. Six imports already did this:
- storm-client Utils used org.apache.commons.io.IOUtils, which reaches storm-client through commons-compress.
- storm-server EquivalenceUtils, SchedulerConfigCache, WorkerTokenManager, CgroupManager and Slot used com.google.common, which reaches storm-server only through the optional auto-service dependency.

They now use the shaded classes, like the other imports in the same modules.

The root pom defines the check-unshaded-imports checkstyle execution once, in pluginManagement, with an IllegalImport rule that reads the unshaded.illegalPkgs property. The shared storm_checkstyle.xml stays unchanged. The execution is bound to the phase in the unshaded.check.phase property, which defaults to none. storm-client and storm-server set it to validate and set their own unshaded.illegalPkgs. A comment next to the relocations in storm-shaded-deps/pom.xml points to these two properties.

The two lists differ:
- storm-client bans all relocated packages.
- commons-io, commons-lang3, commons-collections4, json-smart and snakeyaml are direct dependencies of storm-server, so their relocated packages are allowed there.
- storm-server bans com.google.common and com.google.thirdparty, the com.google packages in the shaded jar, so com.google.auto.service stays allowed.

Both lists ban org.apache.curator, so new Curator code in the main code of storm-client and storm-server has to use the shaded package. This fits the ongoing Curator decoupling.

Other modules are not checked, because storm-core, storm-webapp and the external modules declare some of the relocated packages themselves. They inherit the execution with phase none, so it never runs there. Test code is not checked, because tests use curator-test, which is not shaded. The check covers import statements only, not fully qualified class names in code.

## How was the change tested

Without the fixes, storm-client reports the IOUtils import and storm-server reports the five com.google.common imports, and nothing else. A temporary class in storm-server that imports one class from each relocated package fails on the 15 banned packages and passes on the allowed ones and on auto-service. Temporary imports of the first, the last and two other banned packages fail validate in storm-client and in storm-server. The effective pom of storm-client and storm-server binds check-unshaded-imports to validate with their own package lists. The build with the examples and externals profiles passes validate in every module and runs check-unshaded-imports only in storm-client and storm-server. UtilsTest passes in storm-client. SlotTest, WorkerTokenTest, EquivalenceUtilsTest, TestCgroups, TestResourceAwareScheduler and MultitenantSchedulerTest pass in storm-server.

## AI Disclosure

Human-led contribution. Grok Bot, working for dragonknightk, assisted with this change, including the code, tests, and description. All submitted content was reviewed by a human before submission.

Signed-off-by: dragonknightk <[email protected]>
@dragonknightk

Copy link
Copy Markdown
Author

@rzo1 Thanks for the review! I took both suggestions in this PR: the check-unshaded-imports execution is now defined once in the root pom's pluginManagement, and storm-client and storm-server only set their own package lists through a property. The description now also notes that main code can no longer import org.apache.curator directly. (。•̀ᴗ-)✧

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[STORM-3859] Change mvn build process to validate shaded classes are not used directly

2 participants