We had to @Ignore couple tests and that turned out to be problem for Flank, because it tries to run them anyways (by passing -e class class1#test1,class1#test2,class2#test1) and that crashes android.support.test.runner.AndroidJUnitRunner with following stacktrace:
java.lang.NullPointerException: Attempt to invoke interface method 'int java.lang.CharSequence.length()' on a null object reference
at java.util.regex.Matcher.reset(Matcher.java:995)
at java.util.regex.Matcher.<init>(Matcher.java:174)
at java.util.regex.Pattern.matcher(Pattern.java:1006)
at android.support.test.internal.runner.TestRequestBuilder$MethodFilter.stripParameterizedSuffix(TestRequestBuilder.java:523)
at android.support.test.internal.runner.TestRequestBuilder$MethodFilter.evaluateTest(TestRequestBuilder.java:509)
at android.support.test.internal.runner.TestRequestBuilder$ParentFilter.shouldRun(TestRequestBuilder.java:145)
at android.support.test.internal.runner.TestRequestBuilder$ClassAndMethodFilter.evaluateTest(TestRequestBuilder.java:451)
at android.support.test.internal.runner.TestRequestBuilder$ParentFilter.shouldRun(TestRequestBuilder.java:145)
at org.junit.runner.manipulation.Filter$3.shouldRun(Filter.java:112)
at org.junit.runners.ParentRunner.shouldRun(ParentRunner.java:434)
at org.junit.runners.ParentRunner.filter(ParentRunner.java:382)
at org.junit.runner.manipulation.Filter.apply(Filter.java:97)
at android.support.test.internal.runner.TestRequestBuilder$LenientFilterRequest.getRunner(TestRequestBuilder.java:412)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at android.support.test.internal.runner.TestExecutor.execute(TestExecutor.java:58)
at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:369)
at
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1962)
And of course Firebase marks test run as successful even though no tests were run.
This should be possible to fix now that https://github.com/linkedin/dex-test-parser provides visibility into annotations.
We'll add support for this.
Most helpful comment
This should be possible to fix now that https://github.com/linkedin/dex-test-parser provides visibility into annotations.