Mockk: Feature: produce better exception messages for cases of pre-"Android P" reduced feature set

[Spheniscine]

Issue

MockK-Android can't mock final (non-open) functions or properties using every, however, the regular MockK (unit test) variant works fine. This is true for both dependent and independent functions/properties, but the error messages differ.

I have prepared a minimal working example in this GitHub repo.

Expected Behavior

All tests in both MockkAndroidTest.kt (instrumented) and MockkTest.kt (unit) pass.

Current Behavior

All tests in MockkTest.kt pass. However, only the tests in MockkAndroidTest.kt corresponding to open functions/properties pass.

Failure Information (for bugs)

The tests for functions/properties dependent on the Dependency class return this error:
java.lang.NullPointerException: Attempt to invoke virtual method 'java.lang.String com.github.spheniscine.mockkandroidbug.Dependency.generateAnswer()' on a null object reference

However, the tests for independent functions return this error:
io.mockk.MockKException: Missing calls inside every { ... } block.

Steps to Reproduce

1. Download this GitHub repo.
2. Run the Android instrumented tests and watch half of them fail. Identical unit tests are given for comparison, but they should all pass.

Context

• MockK version: 1.8.13kotlin13
• Working computer OS: Ubuntu 18.04.1 LTS
• Android version: 6.0.1
• Kotlin version: 1.3.10
• JUnit version: 4.12
• Type of test: Android instrumented test; unit tests given for comparison

Stack trace

NullPointerException example:

java.lang.NullPointerException: Attempt to invoke virtual method 'java.lang.String com.github.spheniscine.mockkandroidbug.Dependency.generateAnswer()' on a null object reference
at com.github.spheniscine.mockkandroidbug.ClassUnderTest.getProperty(ClassUnderTest.kt:6)
at com.github.spheniscine.mockkandroidbug.MockkAndroidTest$propertyTest$1.invoke(MockkAndroidTest.kt:16)
at com.github.spheniscine.mockkandroidbug.MockkAndroidTest$propertyTest$1.invoke(MockkAndroidTest.kt:11)
at io.mockk.impl.eval.RecordedBlockEvaluator$record$block$1.invoke(RecordedBlockEvaluator.kt:24)
at io.mockk.impl.eval.RecordedBlockEvaluator$enhanceWithNPERethrow$1.invoke(RecordedBlockEvaluator.kt:74)
at io.mockk.impl.recording.JvmAutoHinter.autoHint(JvmAutoHinter.kt:23)
at io.mockk.impl.eval.RecordedBlockEvaluator.record(RecordedBlockEvaluator.kt:36)
at io.mockk.impl.eval.EveryBlockEvaluator.every(EveryBlockEvaluator.kt:25)
at io.mockk.MockKDsl.internalEvery(API.kt:93)
at io.mockk.MockKKt.every(MockK.kt:104)
at com.github.spheniscine.mockkandroidbug.MockkAndroidTest.propertyTest(MockkAndroidTest.kt:16)
at java.lang.reflect.Method.invoke(Native Method)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at android.support.test.runner.AndroidJUnit4.run(AndroidJUnit4.java:101)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at android.support.test.internal.runner.TestExecutor.execute(TestExecutor.java:56)
at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:384)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1960)

MockKException example

io.mockk.MockKException: Missing calls inside every { ... } block.
at io.mockk.impl.recording.states.StubbingState.checkMissingCalls(StubbingState.kt:14)
at io.mockk.impl.recording.states.StubbingState.recordingDone(StubbingState.kt:8)
at io.mockk.impl.recording.CommonCallRecorder.done(CommonCallRecorder.kt:42)
at io.mockk.impl.eval.RecordedBlockEvaluator.record(RecordedBlockEvaluator.kt:60)
at io.mockk.impl.eval.EveryBlockEvaluator.every(EveryBlockEvaluator.kt:25)
at io.mockk.MockKDsl.internalEvery(API.kt:93)
at io.mockk.MockKKt.every(MockK.kt:104)
at com.github.spheniscine.mockkandroidbug.MockkAndroidTest.indyPropertyTest(MockkAndroidTest.kt:44)
at java.lang.reflect.Method.invoke(Native Method)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at android.support.test.runner.AndroidJUnit4.run(AndroidJUnit4.java:101)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at android.support.test.internal.runner.TestExecutor.execute(TestExecutor.java:56)
at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:384)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1960)

[Spheniscine]

This seems to be a known issue; according to ANDROID.md mocking final classes and properties only works for ≥ Android P.
In which case, the exceptions/error messages have some room for improvement; just from them, I had no idea I needed to open the systems under test, and was about ready to go back to Mockito (which ironically, would require me to open everything anyway)

[oleksiyp]

Yes, seems I need to improve exceptions. No way to make it work before <= Android P. Good thing is that you need only to run it in emulator >= Android P, and do not need migrate the whole project.

[casadogg]

I ran across this problem as well. Wrote a bunch of tests at work and ran them using a device with Android P. Came home, tried to run the tests in another device with Android M, kept getting this error:
io.mockk.MockKException: Missing calls inside every { ... } block.

If I need to test against a device with Android < P, I'd have to open the dependencies, correct?

[oleksiyp]

Sorry, no magic. This interface that makes all this features available apeared since Android P. You need only emulator itself to be >= P, app can have different compatibility setting.

[oleksiyp]

Besides opening many features not working at all. Check http://mockk.io/ANDROID

[oleksiyp]

This issue will be used to improve the error message.

[cuichanghao]

Event if Android P instrumentTest extension function mocking not work correct .
mockk verify{

fun Context.startCustomTab(url: String){
}

[cuichanghao]

Event if Android P instrumentTest extension function mocking not work correct .

Sorry. I resolve this problem with mockkStatic()

[tomwijgers]

So I can't test on 28 or higher because of https://github.com/mockk/mockk/issues/260, and can't test on 27 or lower because of this. I guess it's time for dexOpener.

[dave-kennedy]

I had no idea this page existed. It would be really helpful to provide more visibility for this. I spent about an hour searching before I discovered this issue and the cause of my test failures, assuming this applies to using slot<T> as well.

[v1sd3v]

Any news on this issue? I get this io.mockk.MockKException: Missing calls inside every { ... } block.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. If you are sure that this issue is important and should not be marked as stale just ask to put an important label.

[PabiMoloi]

Still reproducible, I get this io.mockk.MockKException: Missing calls inside every { ... } block.

[geosmart]

Still reproducible, I get this io.mockk.MockKException: Missing calls inside every { ... } block.

+1