Spock: Unable to run tests with maven

[rickle-msft]

I have followed the configurations in the sample project's pom, and maven still skips over spock tests. The plugins I have are as follows. My Spock tests are named *Test and are under src/test/groovy. I am able to run the tests in IntelliJ, but not through maven from the command line. Thank you for your help.

<pluginManagement> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.1</version> <configuration> <source>1.8</source> <target>1.8</target> <showWarnings>true</showWarnings> <showDeprecation>true</showDeprecation> <compilerArgument>-Xlint:unchecked</compilerArgument> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20.1</version> <configuration> <testSourceDirectory>src/test/groovy</testSourceDirectory> <useFile>false</useFile> <includes> <include>**/*Test.*</include> </includes> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-release-plugin</artifactId> <version>2.5.2</version> </plugin> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>build-helper-maven-plugin</artifactId> </plugin> <plugin> <!-- The gmavenplus plugin is used to compile Groovy code. To learn more about this plugin, visit https://github.com/groovy/GMavenPlus/wiki --> <groupId>org.codehaus.gmavenplus</groupId> <artifactId>gmavenplus-plugin</artifactId> <version>1.6</version> <executions> <execution> <goals> <goal>compile</goal> <goal>compileTests</goal> </goals> </execution> </executions> </plugin> </plugins> </pluginManagement>

Java/JDK

1.8

Groovy version

2.4.13

Build tool version

Apache Maven

3.5.2

Operating System

Windows

Build-tool dependencies used

Apache Maven

<dependency>
  <groupId>org.spockframework</groupId>
  <artifactId>spock-core</artifactId>
  <version>1.1-groovy-2.4-</version>
</dependency>

[sdoeringNew]

If you add the plugins to the plugin management, they still have to be added to the plugins of the specific build in order to turn them on.

Please try it like this:

<build>
    <plugins>
        <!-- all the Spock plugins here -->
    </plugins>
</build>

And not like this:

<build>
    <pluginManagement>
        <plugins>
            <!-- all the Spock plugins here -->
        </plugins>
    </pluginMangement>
</build>

[rickle-msft]

This worked! Thank you!

[sonerd]

I have similar issues as described above. Somehow my spock tests are not picked by the
maven-falilsafe-plugin
My setup:

• maven: 3.3.9
• jdk: Jdk:1.8.0_172

This how my pom.xml looks:

<dependency>
<groupId>org.spockframework</groupId>
<artifactId>spock-core</artifactId>
<version>1.1-groovy-2.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
<version>2.4.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.spockframework</groupId>
<artifactId>spock-spring</artifactId>
<version>1.1-groovy-2.4</version>
<scope>test</scope>
</dependency>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<id>integration-test</id>
<phase>integration-test</phase>
<configuration>
<skip>false</skip>
<includes>
<include>**/*Spec.class</include>
</includes>
</configuration>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

I am able to run the tests from my IDE but with maven no chance. The related test classes are compiled as far I can see it in the target folder. I also tried it with

<include>**/*Spec.java</include>
and
<include>**/*Spec.groovy</include>

but no success.

Thanks in advance.

[leonard84]

@sonerd I would recommend to just follow the maven naming conventions, i.e. *Test for Surefire and *IT for failsafe. Otherwise, just to make sure, failsafe tests are only executed during verify and not test.

[sonerd]

@leonard84 Thanks for the feedback. I already had tried to name my spock test as the failsafe plugin expect it but it didn't help. In my current project I have existing JUnit integration tests running fine. I just started to add new integration tests based on spock and want them to also run with maven. The tests are started with:

mvn clean verify -Pintegration.test

Actually I have also tried something like that:

<includes>
    <include>**/*IntegrationTest.class</include> <!-- for starting the existing JUnit based tests -->
    <include>**/*MySpockSpec.class</include>
</includes>

The spock test is picked but not executed as the output says:

Running MySpockSpec
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.268 sec - in MySpockSpec

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[sonerd]

BTW, this is the version of the failsafe plugin I use:
<failsafe.plugin.version>2.19</failsafe.plugin.version>

[leonard84]

Have you tried https://github.com/spockframework/spock-example

[sonerd]

Yes I have tried the sample project and the maven part works smoothly, but the sample includes the surefire plugin and not the failsafe. In case of a greenfield project I would structure it as the sample project probably, but the project I am working on is a legacy project where I want to rewrite the existing tests with spock.

[mkutz]

Here you can find a working project using Failsafe:
https://github.com/mkutz/geb-testing-workshop/blob/master/pom.xml

Maybe it helps you to spot the problem.

Am Mo., 5. Nov. 2018, 15:49 hat Soner Dastan [e-mail]
geschrieben:

Yes I have the sample project and the maven part works smoothly, but the
sample includes the surefire plugin and not the failsafe. In case of a
greenfield project I would structure it as sample project probably, but the
project I am working on is a legacy project where I want to rewrite the
existing tests with spock.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/spockframework/spock/issues/847#issuecomment-435901599,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADh-qk0v0Suwb0o3Gtnnz1a8s6YCycE7ks5usE_lgaJpZM4UK9F8
.

[sonerd]

@mkutz thanks, I will give it a try.

[sonerd]

@mkutz The tests from your project are executed successfully on my machine as expected.
I have taken the same versions as you use, but still struggling.

I am still investigating.

To enclose the problem I have added a very simple integration test into my project

class HelloSpockSpecIntegrationTest extends Specification {

    def "a simple test"() {
        println("######################## Executing Test #########################")
        expect: true
        println("######################## Finished Test #########################")
    }
}

my pom.xml:
<include>**/HelloSpockSpecIntegrationTest.class</include>

After running the test with mvn clean verify this is printed in my console:

[INFO] HelloSpockSpecIntegrationTest
######################## Executing Test #########################
######################## Finished Test #########################
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.094 s - in HelloSpockSpecIntegrationTest
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO] --- maven-failsafe-plugin:2.22.0:verify (integration-test) ---
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:52 min
[INFO] Finished at: 2018-11-06T12:20:05+01:00
[INFO] Final Memory: 93M/1317M
[INFO] ------------------------------------------------------------------------

It seems that the test is executed but the summary doesn't show it.
When I change the test to make it fail:

class HelloSpockSpecIntegrationTest extends Specification {

    def "a simple test"() {
        println("######################## Executing Test #########################")
        expect: 1 == 2
        println("######################## Finished Test #########################")
    }
}

The console:

[INFO] Running HelloSpockSpecIntegrationTest
######################## Executing Test #########################
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.192 s - in HelloSpockSpecIntegrationTest
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO] --- maven-failsafe-plugin:2.22.0:verify (integration-test)
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:48 min
[INFO] Finished at: 2018-11-06T12:22:05+01:00
[INFO] Final Memory: 93M/1317M
[INFO] ------------------------------------------------------------------------

The outcome is different, as the second println is not executed, but why is the summary still the same as above and the build is still successful?

[mkutz]

@sonerd That is really strange. The test definitely failed, otherwise the second println would be in your logs. However failsafe ignores it apparently. However, the project I sent you is executed by Travis and there is a proper test report (see https://travis-ci.org/mkutz/geb-testing-workshop).

[sonerd]

Finally I solved the problem. After debugging the maven plugin itself (added as project dependency to my project) I saw that maven was using TestNG as provider: AbstractSurefireMojo

The version of TestNG used in my case was 6.8.. In my case this version is set inside a parent pom of my project.

To double check that TestNG is not the troublemaker I debugged the project of @mkutz including the maven plugin (added as project dependency) and saw that TestNG was used there too, but in version 6.13.1, as this project is not including TestNG dependency explicitly.

So I changed the version in my parent pom to 6.13.1 and finally everything worked as I expected.
I don't know why the older version of TestNG was not working properly.

Thanks for the support.