Kotest: Robolectric Extension fails with "this method should only be called by Robolectric"

Created on 10 Aug 2020  路  9Comments  路  Source: kotest/kotest

I'm having issues with running kotest in conjunction with Robolectric.

  • Android Studio 4.0.1
  • Kotest Plugin version 1.1.0-SNAPSHOT-IC-2019.3 (due to #1601)

_build.gradle_:

apply plugin: 'com.android.application'
apply plugin: 'kotlin-android'
apply plugin: 'kotlin-android-extensions'

android {
    compileSdkVersion 30
    buildToolsVersion "30.0.1"

    defaultConfig {
        applicationId "com.averageatmost.kotest_junit4"
        minSdkVersion 23
        targetSdkVersion 29
        versionCode 1
        versionName "1.0"
    }

    testOptions {
        unitTests {
            includeAndroidResources = true
        }
    }
}

dependencies {
    implementation fileTree(dir: "libs", include: ["*.jar"])
    implementation "org.jetbrains.kotlin:kotlin-stdlib:1.3.72"
    implementation 'androidx.core:core-ktx:1.3.0'
    implementation 'androidx.appcompat:appcompat:1.2.0-rc01'
    implementation 'androidx.constraintlayout:constraintlayout:2.0.0-beta8'
    testImplementation("io.kotest:kotest-runner-junit4-jvm:4.1.3")
    testImplementation("io.kotest:kotest-runner-console-jvm:4.1.3")
    testImplementation("io.kotest:kotest-assertions-core-jvm:4.1.3")
    testImplementation("org.robolectric:robolectric:4.4-alpha-3")
    testImplementation("io.kotest:kotest-extensions-robolectric-jvm:4.0.1")
}

_Robolectric.kt_:

package com.averageatmost.kotest_junit4

import io.kotest.core.config.AbstractProjectConfig
import io.kotest.core.extensions.Extension
import io.kotest.core.spec.style.FeatureSpec
import io.kotest.experimental.robolectric.RobolectricExtension
import io.kotest.experimental.robolectric.RobolectricTest
import io.kotest.matchers.shouldBe


class MyProjectLevelConfig : AbstractProjectConfig() {
    override fun extensions(): List<Extension> = super.extensions() + RobolectricExtension()
}

@RobolectricTest
class ExampleUnitTest : FeatureSpec({
    feature("Addition") {
        scenario("is correct") {
            2 + 2 shouldBe 4
        }
    }
})

I have also added a _robolectric.properties_ file to app/src/test/resources with sdk=23 to avoid having to run my tests in Java 9.

When running the test inside the IDE, I get the following error:

java.lang.IllegalStateException: this method should only be called by Robolectric
    at org.robolectric.shadows.ShadowDisplayManager.configureDefaultDisplay(ShadowDisplayManager.java:69)
    at org.robolectric.android.Bootstrap.setUpDisplay(Bootstrap.java:18)
    at org.robolectric.android.internal.AndroidTestEnvironment.installAndCreateApplication(AndroidTestEnvironment.java:219)
    at org.robolectric.android.internal.AndroidTestEnvironment.setUpApplicationState(AndroidTestEnvironment.java:171)
    at org.robolectric.RobolectricTestRunner.beforeTest(RobolectricTestRunner.java:302)
    at io.kotest.experimental.robolectric.ContainedRobolectricRunner.containedBefore(ContainedRobolectricRunner.kt:26)
    at io.kotest.experimental.robolectric.RobolectricExtension.beforeTest(RobolectricExtension.kt:39)
    at io.kotest.experimental.robolectric.RobolectricExtension.intercept(RobolectricExtension.kt:31)
    at io.kotest.core.runtime.TestCaseExecutor.intercept(TestCaseExecutor.kt:70)
    at io.kotest.core.runtime.TestCaseExecutor.execute(TestCaseExecutor.kt:50)
    at io.kotest.core.engine.SingleInstanceSpecRunner.runTest(SingleInstanceSpecRunner.kt:61)
    at io.kotest.core.engine.SingleInstanceSpecRunner$Context.registerTestCase(SingleInstanceSpecRunner.kt:39)
    at io.kotest.core.runtime.TestCaseExecutor$executeAndWait$2$1$1$3$1$contextp$1.registerTestCase(TestCaseExecutor.kt:192)
    at io.kotest.core.test.TestContext.registerTestCase(TestContext.kt:41)
    at io.kotest.core.spec.style.scopes.ContainerScope$DefaultImpls.addTest(ContainerScope.kt:50)
    at io.kotest.core.spec.style.scopes.FeatureScope.addTest(FeatureScope.kt:22)
    at io.kotest.core.spec.style.scopes.ContainerScope$DefaultImpls.addContainerTest(ContainerScope.kt:35)
    at io.kotest.core.spec.style.scopes.FeatureScope.addContainerTest(FeatureScope.kt:22)
    at io.kotest.core.spec.style.scopes.FeatureScope.scenario(FeatureScope.kt:57)
    at com.averageatmost.kotest_junit4.ExampleUnitTest$1$1.invokeSuspend(ExampleUnitTest.kt:18)
    at com.averageatmost.kotest_junit4.ExampleUnitTest$1$1.invoke(ExampleUnitTest.kt)
    at io.kotest.core.spec.style.scopes.FeatureSpecScope$addFeature$1.invokeSuspend(FeatureSpecScope.kt:20)
    at io.kotest.core.spec.style.scopes.FeatureSpecScope$addFeature$1.invoke(FeatureSpecScope.kt)
    at io.kotest.core.runtime.ExecutionsKt$executeWithBehaviours$2$1.invokeSuspend(executions.kt:16)
    at io.kotest.core.runtime.ExecutionsKt$executeWithBehaviours$2$1.invoke(executions.kt)
    at io.kotest.core.runtime.ExecutionsKt.wrapTestWithGlobalAssert(executions.kt:43)
    at io.kotest.core.runtime.ExecutionsKt$executeWithBehaviours$2.invokeSuspend(executions.kt:15)
    at io.kotest.core.runtime.ExecutionsKt$executeWithBehaviours$2.invoke(executions.kt)
    at io.kotest.core.runtime.ExecutionsKt.wrapTestWithAssertionModeCheck(executions.kt:27)
    at io.kotest.core.runtime.ExecutionsKt.executeWithBehaviours(executions.kt:14)
    at io.kotest.core.runtime.TestCaseExecutor$executeAndWait$2$1$1$3$1.invokeSuspend(TestCaseExecutor.kt:195)
    at io.kotest.core.runtime.TestCaseExecutor$executeAndWait$2$1$1$3$1.invoke(TestCaseExecutor.kt)
    at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:91)
    at kotlinx.coroutines.CoroutineScopeKt.coroutineScope(CoroutineScope.kt:189)
    at io.kotest.core.runtime.TestCaseExecutor$executeAndWait$2$1$1$3.invokeSuspend(TestCaseExecutor.kt:189)
    at io.kotest.core.runtime.TestCaseExecutor$executeAndWait$2$1$1$3.invoke(TestCaseExecutor.kt)
    at io.kotest.core.runtime.ReplayKt.replay(replay.kt:19)
    at io.kotest.core.runtime.TestCaseExecutor$executeAndWait$2$1$1.invokeSuspend(TestCaseExecutor.kt:184)
    at io.kotest.core.runtime.TestCaseExecutor$executeAndWait$2$1$1.invoke(TestCaseExecutor.kt)
    at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturnIgnoreTimeout(Undispatched.kt:102)
    at kotlinx.coroutines.TimeoutKt.setupTimeout(Timeout.kt:111)
    at kotlinx.coroutines.TimeoutKt.withTimeout(Timeout.kt:32)
    at io.kotest.core.runtime.TestCaseExecutor$executeAndWait$2$1.invokeSuspend(TestCaseExecutor.kt:183)
    at io.kotest.core.runtime.TestCaseExecutor$executeAndWait$2$1.invoke(TestCaseExecutor.kt)
    at io.kotest.core.runtime.ExecutorExecutionContext$executeWithTimeoutInterruption$$inlined$suspendCoroutine$lambda$2.invokeSuspend(ExecutorExecutionContext.kt:47)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
    at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
    at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:79)
    at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:54)
    at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
    at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:36)
    at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
    at io.kotest.core.runtime.ExecutorExecutionContext.executeWithTimeoutInterruption-D5N0EJY(ExecutorExecutionContext.kt:46)
    at io.kotest.core.runtime.TestCaseExecutor$executeAndWait$2.invokeSuspend(TestCaseExecutor.kt:182)
    at io.kotest.core.runtime.TestCaseExecutor$executeAndWait$2.invoke(TestCaseExecutor.kt)
    at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturnIgnoreTimeout(Undispatched.kt:102)
    at kotlinx.coroutines.TimeoutKt.setupTimeout(Timeout.kt:111)
    at kotlinx.coroutines.TimeoutKt.withTimeout(Timeout.kt:32)
    at io.kotest.core.runtime.TestCaseExecutor.executeAndWait(TestCaseExecutor.kt:180)
    at io.kotest.core.runtime.TestCaseExecutor.invokeTestCase(TestCaseExecutor.kt:149)
    at io.kotest.core.runtime.TestCaseExecutor.executeActiveTest(TestCaseExecutor.kt:118)
    at io.kotest.core.runtime.TestCaseExecutor$intercept$2.invokeSuspend(TestCaseExecutor.kt:69)
    at io.kotest.core.runtime.TestCaseExecutor$intercept$2.invoke(TestCaseExecutor.kt)
    at io.kotest.core.runtime.TestCaseExecutor.executeIfActive(TestCaseExecutor.kt:83)
    at io.kotest.core.runtime.TestCaseExecutor.intercept(TestCaseExecutor.kt:69)
    at io.kotest.core.runtime.TestCaseExecutor$intercept$3.invokeSuspend(TestCaseExecutor.kt:70)
    at io.kotest.core.runtime.TestCaseExecutor$intercept$3.invoke(TestCaseExecutor.kt)
    at io.kotest.experimental.robolectric.RobolectricExtension.intercept(RobolectricExtension.kt:32)
    at io.kotest.core.runtime.TestCaseExecutor.intercept(TestCaseExecutor.kt:70)
    at io.kotest.core.runtime.TestCaseExecutor.execute(TestCaseExecutor.kt:50)
    at io.kotest.core.engine.SingleInstanceSpecRunner.runTest(SingleInstanceSpecRunner.kt:61)
    at io.kotest.core.engine.SingleInstanceSpecRunner$execute$2$invokeSuspend$$inlined$invoke$lambda$1.invokeSuspend(SingleInstanceSpecRunner.kt:71)
    at io.kotest.core.engine.SingleInstanceSpecRunner$execute$2$invokeSuspend$$inlined$invoke$lambda$1.invoke(SingleInstanceSpecRunner.kt)
    at io.kotest.core.engine.SpecRunner$runParallel$$inlined$map$lambda$2$1.invokeSuspend(SpecRunner.kt:78)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
    at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
    at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:79)
    at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:54)
    at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
    at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:36)
    at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
    at io.kotest.core.engine.SpecRunner$runParallel$$inlined$map$lambda$2.run(SpecRunner.kt:77)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

Process finished with exit code 255

this method should only be called by Robolectric

At first I though this was only an issue with the IDE plugin, as _./gradlew test_ ran just fine, but then I discovered that the tests don't seem to be run at all.

Coding the same test without Kotest is running just fine:

package com.averageatmost.kotest_junit4

import io.kotest.matchers.shouldBe
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.RobolectricTestRunner

@RunWith(RobolectricTestRunner::class)
class RobolectricTest {

    @Test
    fun addition_isCorrect() {
        2 + 2 shouldBe 5
    }
}

Am I missing something?

bug framework android
Source
picture Bombo
👍1

Most helpful comment

Having the same issue with plugin version 1.1.16-IC-2019.3

picture IsLery on 23 Sep 2020
👍3

All 9 comments

Still happening with plugin version 1.1.9-IC-2019.3

Bombo picture Bombo on 11 Aug 2020

This issue happens because of initializing Robolectric twice. It is related to which isolationMode you use, it definitely happens for the InstancePerLead one. We have also faced it, and I am working on a fix for this, gonna open a PR soon.

BaloghTamas picture BaloghTamas on 17 Aug 2020
👍3

@BaloghTamas Sounds good, although I have a test project with only one FeatureSpec in it (see above) and I'm using the default IsolationMode, which then should be SingleInstance, right?

Maybe your fix takes care of this, still, but the reason you gave sounds like a different issue.

Bombo picture Bombo on 17 Aug 2020

Still happening with plugin version 1.1.11-IC-2019.3

Bombo picture Bombo on 20 Aug 2020

Having the same issue with plugin version 1.1.16-IC-2019.3

IsLery picture IsLery on 23 Sep 2020
👍3

Still the same issue, changing the isolationMode makes no difference.

AhmedMourad0 picture AhmedMourad0 on 4 Oct 2020

Are there no workarounds to this?

AhmedMourad0 picture AhmedMourad0 on 5 Oct 2020

Still happening with:
Plugin: 1.1.20-IC-2020.1
Kotest: 4.3.1 (Console Runner 4.1.3.2)

Bombo picture Bombo on 25 Nov 2020

I'm having the same problem, are there any fixes for this problem?

victor5171 picture victor5171 on 17 Dec 2020
👍2
Was this page helpful?
0 / 5 - 0 ratings

Related issues

sksamuel picture sksamuel  路  3Comments
Reevn picture Reevn  路  5Comments
JoonasC picture JoonasC  路  4Comments
myuwono picture myuwono  路  8Comments
r3mi picture r3mi  路  7Comments