Kotest: wrong count of tests

[priyanshus]

Running below tests shows me total test count as 3 and failed test as 2.
com.x.HelloWorldTest > test2 FAILED
java.lang.AssertionError at HelloWorldTest.kt:17
Caused by: java.lang.AssertionError at HelloWorldTest.kt:17

com.x.HelloWorldTest > executionError FAILED
java.lang.AssertionError at HelloWorldTest.kt:17
Caused by: java.lang.AssertionError at HelloWorldTest.kt:17

3 tests completed, 2 failed

class HelloWorldTest: AnnotationSpec(){
    @Test
    fun test1() {
        1 shouldBe 1
    }

    @Test
    fun test2() {
        3 shouldBe 2
    }
}

Using
testImplementation 'io.kotest:kotest-runner-junit5-jvm:4.0.1'
testImplementation 'io.kotest:kotest-assertions-core-jvm:4.0.1'
JDK 11
GRADLE 5.5.1

[ashishkujoy]

@sksamuel Its happening after https://github.com/kotest/kotest/commit/b1820a70425490f14ef55663831f9c3bd7f804c4

[sksamuel]

It was probably always happening.
It's just the way gradle reports things, it doesn't really support nested
tests.
There's not a lot we can do, short of writing our own gradle plugin.

On Tue, 31 Mar 2020 at 11:25, ashish kumar joy [e-mail]
wrote:

@sksamuel https://github.com/sksamuel Its happening after b1820a7
https://github.com/kotest/kotest/commit/b1820a70425490f14ef55663831f9c3bd7f804c4

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kotest/kotest/issues/1330#issuecomment-606733181, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAFVSGTVY7ZCBDYVEPDS3ODRKIKPPANCNFSM4LXP5JCA
.

[crazyk2]

@sksamuel FYI..The above issue can be reproduced without gradle as well. Running the tests through IntelliJ shows same result.

How does IntelliJ run test?

[sksamuel]

Try 4.0.2

[low205]

Same issue with 4.0.2, gradle 6.3, jdk 11.

[riccardo-felipe]

I don't now if it is related, but seems to be since a similar problem occurs when executing following test:

class KoTestBDDTest : BehaviorSpec({
    Given("an UUID") {
        and("a number") {
           `when`("needed") {
                then("check it") {
                    val uuid = UUID.randomUUID()
                    uuid.toString().shouldBeUUID()
                }
            }
        }
    }
})

Running via IntelliJ IDEA Kotest plugin: 1 test is passing
Running via maven surefire: 4 are passing
Running via gradle: 4 tests are passing

[sksamuel]

Intellij does it properly. The others just assume each node is a test.

On Mon, 29 Jun 2020 at 10:00, riccardo-felipe [e-mail]
wrote:

I don't now if it is related, but seems to be since a similar problem
occurs when executing following test:

class KoTestBDDTest : BehaviorSpec({
Given("an UUID") {
and("a number") {
when("needed") {
then("check it") {
val uuid = UUID.randomUUID()
uuid.toString().shouldBeUUID()
}
}
}
}
})

Running via IntelliJ IDEA Kotest plugin: 1 test is passing
Running via maven surefire: 4 are passing
Running via gradle: 4 tests are passing

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kotest/kotest/issues/1330#issuecomment-651177627, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAFVSGVVGULNRSMOJAXLI7LRZCUBHANCNFSM4LXP5JCA
.

[sksamuel]

I don't now if it is related, but seems to be since a similar problem occurs when executing following test:

Running via IntelliJ IDEA Kotest plugin: 1 test is passing
Running via maven surefire: 4 are passing
Running via gradle: 4 tests are passing

So the plugin is written by the kotest team and understands arbitrarily nested tests.
The maven / gradle plugins use junit platform, which doesn't.

If you want to try the pre-release kotest gradle plugin you can.

https://plugins.gradle.org/plugin/io.kotest

Just add that to your build and run ./gradlew kotest
It will only work for JVM builds.

[ashishkujoy]

Fixed in master by #1655