Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,15 +91,16 @@ therefore need `-XstartOnFirstThread`. There are two contexts:

### One JUnit run per test module

Every test bundle (packaging `eclipse-test-plugin`) runs its own tests in its own Eclipse runtime. `tycho-surefire-plugin` is configured once in `ddk-parent/pom.xml`; individual test poms carry no test configuration. Surefire finds the tests by class name, so there are no test suites to maintain. Fragment test bundles work the same way, with their host in the runtime.
Every test bundle (packaging `eclipse-test-plugin`) runs its own tests in its own Eclipse runtime. `tycho-surefire-plugin` is configured once in `ddk-parent/pom.xml`; individual test poms only set the properties below if they need a workbench. Surefire finds the tests by class name, so there are no test suites to maintain. Fragment test bundles work the same way, with their host in the runtime.

**Consequences for agents:**

- A new test needs no registration: name the class `*Test`, `*Tests`, `Test*` or `*TestCase`. Classes with other names are not run; CI fails if a concrete class declaring tests has no report (`.github/scripts/check-surefire-reports.sh`).
- A new test bundle needs packaging `eclipse-test-plugin` and an entry in the `ddk-parent/pom.xml` modules list. With packaging `eclipse-plugin` its tests do not run.
- A module's runtime holds only the dependency closure of its manifest. If a test needs a bundle at runtime, for example a language's UI bundle, declare it in that test bundle's `META-INF/MANIFEST.MF`.
- Test classes run in alphabetical order within a module and must not depend on running after other classes.
- With `-T`, test runtimes run one at a time (`reactorConcurrencyLevel` in `ddk-parent/pom.xml`), while everything else stays parallel. `CheckQuickfixTest` failed intermittently on macOS when they overlapped.
- Test modules run headless by default, which is Tycho's default: no workbench starts, so `PlatformUI.getWorkbench()` and any Guice injector that needs `IWorkbench` fail. A module that needs a workbench sets `tycho.surefire.useUIHarness` to `true` and `test.reactorConcurrencyLevel` to `1` in its pom's `<properties>`.
- With `-T`, headless modules run in parallel while modules with a workbench take turns (`test.reactorConcurrencyLevel`). `CheckQuickfixTest` has failed intermittently on macOS when workbenches overlapped; rerun serially before debugging such a failure.
- A failed module's tests leave their log in `<module>/target/work/data/.metadata/.log`. CI archives these logs for failed builds.
- To run the tests of one UI bundle from Eclipse, use its launch configuration (e.g. `com.avaloq.tools.ddk.check.ui.test/CheckUiTests.launch`), which runs all tests of the project via JUnit discovery in Eclipse.

Expand Down
5 changes: 5 additions & 0 deletions com.avaloq.tools.ddk.check.core.test/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,9 @@

<artifactId>com.avaloq.tools.ddk.check.core.test</artifactId>
<packaging>eclipse-test-plugin</packaging>

<properties>
<tycho.surefire.useUIHarness>true</tycho.surefire.useUIHarness>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something is off here, this is a core.test plugin, so it should not need UI. Maybe some test need a bit of rework to make this possible?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, both modules are UI tests in practice. I looked into what it takes to run them headless:

  • Every headless failure has one cause: the Check UI injector cannot be created without a workbench ("No implementation for IWorkbench was bound", requested by Xtext's MarkerResolutionGenerator).
  • check.core.test: 9 classes use the UI injector directly, and AbstractCheckTestCase uses it for 6 more that need a plug-in project in the workspace. Headless, 69 of 72 tests error.
  • checkcfg.core.test: resolving a Check catalog loads the .check resources through the Check UI, so 6 of 11 tests fail headless.

Proposed rework, all in test bundles:

  1. Switch 6 check.core.test classes to the runtime injector. I tried this locally: their 63 tests pass headless.
  2. Move AbstractCheckTestCase, AbstractCheckGenerationTestCase and their 6 subclasses to check.ui.test. check.core.test is then headless and loses the opt-in.
  3. Rename checkcfg.core.test to checkcfg.ui.test, since its tests cover content assist, a workspace project and catalogs contributed by plug-ins.

I would do this in a follow-up PR instead of here. This PR only changes how the harness is configured, while the rework moves about 1100 lines of tests between bundles and deserves its own review. The follow-up would remove both opt-ins. Would that work for you, or do you prefer the rework to land first?

This also explains the ConcurrentModificationException listed under "Known limits": it only happens headless, when the Check UI injector fails while CheckCfgUtil.getAllLanguages() iterates the registry. I will update the description.

Posted by Claude Code on behalf of @joaodinissf.

<test.reactorConcurrencyLevel>1</test.reactorConcurrencyLevel>
</properties>
</project>
5 changes: 5 additions & 0 deletions com.avaloq.tools.ddk.check.test.runtime.tests/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,9 @@

<artifactId>com.avaloq.tools.ddk.check.test.runtime.tests</artifactId>
<packaging>eclipse-test-plugin</packaging>

<properties>
<tycho.surefire.useUIHarness>true</tycho.surefire.useUIHarness>
<test.reactorConcurrencyLevel>1</test.reactorConcurrencyLevel>
</properties>
</project>
5 changes: 5 additions & 0 deletions com.avaloq.tools.ddk.check.ui.test/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,9 @@

<artifactId>com.avaloq.tools.ddk.check.ui.test</artifactId>
<packaging>eclipse-test-plugin</packaging>

<properties>
<tycho.surefire.useUIHarness>true</tycho.surefire.useUIHarness>
<test.reactorConcurrencyLevel>1</test.reactorConcurrencyLevel>
</properties>
</project>
5 changes: 5 additions & 0 deletions com.avaloq.tools.ddk.checkcfg.core.test/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,9 @@

<artifactId>com.avaloq.tools.ddk.checkcfg.core.test</artifactId>
<packaging>eclipse-test-plugin</packaging>

<properties>
<tycho.surefire.useUIHarness>true</tycho.surefire.useUIHarness>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something is off here, this is a core.test plugin, so it should not need UI. Maybe some test need a bit of rework to make this possible?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Answered in the thread on check.core.test/pom.xml; the plan there covers this module too (rename to checkcfg.ui.test).

Posted by Claude Code on behalf of @joaodinissf.

<test.reactorConcurrencyLevel>1</test.reactorConcurrencyLevel>
</properties>
</project>
5 changes: 5 additions & 0 deletions com.avaloq.tools.ddk.sample.helloworld.ui.test/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,9 @@
<groupId>com.avaloq.tools.ddk</groupId>
<artifactId>com.avaloq.tools.ddk.sample.helloworld.ui.test</artifactId>
<packaging>eclipse-test-plugin</packaging>

<properties>
<tycho.surefire.useUIHarness>true</tycho.surefire.useUIHarness>
<test.reactorConcurrencyLevel>1</test.reactorConcurrencyLevel>
</properties>
</project>
5 changes: 5 additions & 0 deletions com.avaloq.tools.ddk.test.ui.test/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,9 @@

<artifactId>com.avaloq.tools.ddk.test.ui.test</artifactId>
<packaging>eclipse-test-plugin</packaging>

<properties>
<tycho.surefire.useUIHarness>true</tycho.surefire.useUIHarness>
<test.reactorConcurrencyLevel>1</test.reactorConcurrencyLevel>
</properties>
</project>
5 changes: 5 additions & 0 deletions com.avaloq.tools.ddk.xtext.ui.test/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,9 @@

<artifactId>com.avaloq.tools.ddk.xtext.ui.test</artifactId>
<packaging>eclipse-test-plugin</packaging>

<properties>
<tycho.surefire.useUIHarness>true</tycho.surefire.useUIHarness>
<test.reactorConcurrencyLevel>1</test.reactorConcurrencyLevel>
</properties>
</project>
5 changes: 3 additions & 2 deletions ddk-parent/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@
<!-- test parameter-->
<test.javaOptions>${runtime.javaOptions}</test.javaOptions>
<test.timeout>1800</test.timeout>
<!-- A test module that needs a workbench sets this to 1, and tycho.surefire.useUIHarness to true. -->
<test.reactorConcurrencyLevel>0</test.reactorConcurrencyLevel>

<!-- maven enforced version: Maven 3.10 rejects the nested-jar dependencies Tycho injects (eclipse-tycho/tycho#6399) -->
<maven-enforcer-rule.version>[3.9.0,3.10)</maven-enforcer-rule.version>
Expand Down Expand Up @@ -273,8 +275,7 @@
<providerHint>junit6</providerHint>
<failIfNoTests>false</failIfNoTests>
<useUIThread>false</useUIThread>
<useUIHarness>true</useUIHarness>
<reactorConcurrencyLevel>1</reactorConcurrencyLevel> <!-- one test runtime at a time under -T -->
<reactorConcurrencyLevel>${test.reactorConcurrencyLevel}</reactorConcurrencyLevel>
<forkedProcessTimeoutInSeconds>${test.timeout}</forkedProcessTimeoutInSeconds>
<argLine>-Dslf4j.internal.verbosity=ERROR -Dlogback.configurationFile="${runtime.logbackConfig}" ${test.javaOptions}</argLine>
<appArgLine>-pluginCustomization ${runtime.pluginCustomization}</appArgLine>
Expand Down