Kiwix-android: CLEANUP Tests failed

[julianharty]

I'm opening this issue while I find out why several of the current Instrumentation based tests fail. They might be related to my device, or more general. I'm on the master branch.

I'm running ./gradlew connectedAndroidTest

connectedAndroidTest - Installs and runs instrumentation tests for all flavors on connected devices.

I expect all the tests to pass. However 4 of the 9 tests fail on my Samsung Galaxy Core Prime device.

Starting 9 tests on SM-G361F - 5.1.1

org.kiwix.kiwixmobile.tests.DownloadTest > downloadTest[SM-G361F - 5.1.1] FAILED 
        android.support.test.espresso.PerformException: Error performing 'load adapter data' on view 'Animations or transitions are enabled on the target device.
        For more info check: http://goo.gl/qVu1yV

org.kiwix.kiwixmobile.tests.NetworkTest > networkTest[SM-G361F - 5.1.1] FAILED 
        android.support.test.espresso.NoMatchingViewException: No views in hierarchy found matching: (Child at position 0 in parent with id: org.kiwix.kiwixmobile:id/library_list and is displayed on the screen to the user)
        If the target view is not part of the view hierarchy, you may need to use Espresso.onData to load it from one of the following AdapterViews:android.widget.ListView{45e5555 V.ED.VC. ......ID 0,0-480,0 #7f0f0094 app:id/zim_downloader_list}

org.kiwix.kiwixmobile.utils.BasicTest > basicTest[SM-G361F - 5.1.1] FAILED 
        android.support.test.espresso.NoMatchingViewException: No views in hierarchy found matching: (with class name: is "android.widget.ImageButton" and has parent matching: (with id: 2131689583 and has parent matching: with id: 2131689582) and is displayed on the screen to the user)


org.kiwix.kiwixmobile.utils.ContentTest > contentTest[SM-G361F - 5.1.1] FAILED 
        java.lang.NoSuchMethodError: No static method preGrantStorage()V in class Lorg/kiwix/kiwixmobile/testutils/TestUtils; or its super classes (declaration of 'org.kiwix.kiwixmobile.testutils.TestUtils' appears in /data/app/org.kiwix.kiwixmobile.test-1/base.apk)
        at org.kiwix.kiwixmobile.utils.ContentTest.permissions(ContentTest.java:42)
:app:connectedKiwixDebugAndroidTest FAILED

FAILURE: Build failed with an exception.

I've attached the test report files (zipped).
I'll continue exploring and also run the tests on various devices. I'll update this issue with my progress and findings.

test-results-on-SM-G361F.zip

[julianharty]

I've disabled animations as per
http://google.github.io/android-testing-support-library/docs/espresso/setup/index.html#setup-your-test-environment

Still 4 of the 9 tests fail. I'll keep looking...

[julianharty]

The first clue: The device wasn't connected to the Internet. When I connected the device to a local WiFi two of the failing tests now pass. Let's see if we can find a way to check the network state as part of running the tests.

The following tests still fail.

Starting 9 tests on SM-G361F - 5.1.1

org.kiwix.kiwixmobile.utils.BasicTest > basicTest[SM-G361F - 5.1.1] FAILED 
        android.support.test.espresso.NoMatchingViewException: No views in hierarchy found matching: (with class name: is "android.widget.ImageButton" and has parent matching: (with id: 2131689583 and has parent matching: with id: 2131689582) and is displayed on the screen to the user)

org.kiwix.kiwixmobile.utils.ContentTest > contentTest[SM-G361F - 5.1.1] FAILED 
        java.lang.NoSuchMethodError: No static method preGrantStorage()V in class Lorg/kiwix/kiwixmobile/testutils/TestUtils; or its super classes (declaration of 'org.kiwix.kiwixmobile.testutils.TestUtils' appears in /data/app/org.kiwix.kiwixmobile.test-1/base.apk)
        at org.kiwix.kiwixmobile.utils.ContentTest.permissions(ContentTest.java:42)
:app:connectedKiwixDebugAndroidTest FAILED

FAILURE: Build failed with an exception.

[mhutti1]

The first one fails because you need to be on the welcome page on app start
(get to the page and exit the app then start the test). The second one is
weird.

On 21 Jul 2017 4:25 pm, "Julian Harty" [e-mail] wrote:

The first clue: The device wasn't connected to the Internet. When I
connected the device to a local WiFi two of the failing tests now pass.
Let's see if we can find a way to check the network state as part of
running the tests.

The following tests still fail.

Starting 9 tests on SM-G361F - 5.1.1

org.kiwix.kiwixmobile.utils.BasicTest > basicTest[SM-G361F - 5.1.1] FAILED
android.support.test.espresso.NoMatchingViewException: No views in hierarchy found matching: (with class name: is "android.widget.ImageButton" and has parent matching: (with id: 2131689583 and has parent matching: with id: 2131689582) and is displayed on the screen to the user)

org.kiwix.kiwixmobile.utils.ContentTest > contentTest[SM-G361F - 5.1.1] FAILED
java.lang.NoSuchMethodError: No static method preGrantStorage()V in class Lorg/kiwix/kiwixmobile/testutils/TestUtils; or its super classes (declaration of 'org.kiwix.kiwixmobile.testutils.TestUtils' appears in /data/app/org.kiwix.kiwixmobile.test-1/base.apk)
at org.kiwix.kiwixmobile.utils.ContentTest.permissions(ContentTest.java:42)
:app:connectedKiwixDebugAndroidTest FAILED

FAILURE: Build failed with an exception.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/kiwix/kiwix-android/issues/213#issuecomment-317031367,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACLlWTC10qEZfAmShha2ufJgJck27clIks5sQML8gaJpZM4Offem
.

[mhutti1]

That function was removed a long time ago. Are you sure you have got master update?

https://github.com/kiwix/kiwix-android/commit/fd568a06a46e4b664d1cf0af1411b33caa95108f

[julianharty]

@mhutti1 Thanks for the info. I'm currently trying various devices to see how the tests perform (as presumably we'd like them to run successfully on any/all of the suitable devices).

• So far, I've spent several hours in a losing battle with the infamous 'Overlay detected' on a brand-new Samsung S7 (non-Edge) device with Android 6.0.1 installed. None of the various 'solutions' work (e.g. disabling the 20+ standard and system apps that could draw on screen), checking that one-handed use wasn't enabled, etc. The only way I could get Kiwix to work (independently of the tests) was to allow the file access permission from the Apps menu.

• My very old Galaxy Nexus, running Android 4.3, ran 7 of the 9 tests successfully. The 2 failures are BasicTest and ContentTest. I'm currently running Kiwix by hand to see if I can reproduce the reported behaviour.

I'll keep going, this isn't an urgent task, more one I'd like to help with in the coming few weeks, to help establish a robust set of GUI-based tests for the app.

[mhutti1]

@julianharty I have already mentioned what I think the problems with the galaxy nexus are but the s7 is very weird. We test with an s7 running 6.0.1 on testdroid with no issues.

[julianharty]

On a Samsung Galaxy GT-I9300 7 of the 9 tests failed when the device was in flight mode. However the problems were generally finding elements in the view rather than related to the lack of network connectivity. The menu items are displayed near the bottom of the screen when the tests were running. The same tests failed when the device had a network connection.

For some reason I don't yet know, the menu appears at the bottom centre of the screen while the tests are running. However in the debug build of Kiwix-Android they appear as expected below the vertical 3 dots near the top right of the UI. I'll upload a screenshot shortly (waiting for it to be backed-up from the device).

Starting 9 tests on GT-I9300 - 4.3

org.kiwix.kiwixmobile.tests.BasicTest > testRightDrawer[GT-I9300 - 4.3] FAILED 
        android.support.test.espresso.NoMatchingViewException: No views in hierarchy found matching: with id: org.kiwix.kiwixmobile:id/drawer_layout
        If the target view is not part of the view hierarchy, you may need to use Espresso.onData to load it from one of the following AdapterViews:android.support.v7.view.menu.ExpandedMenuView{4279a188 VFED.VC. .F...... 0,0-592,387 #7f0f005b app:id/expanded_menu}

org.kiwix.kiwixmobile.tests.ContentTest > contentTest[GT-I9300 - 4.3] FAILED 
        android.support.test.espresso.NoMatchingViewException: No views in hierarchy found matching: (with id: org.kiwix.kiwixmobile:id/get_content_card and with text: is "Get Content")
        If the target view is not part of the view hierarchy, you may need to use Espresso.onData to load it from one of the following AdapterViews:android.support.v7.view.menu.ExpandedMenuView{42960b00 VFED.VC. .F...... 0,0-592,387 #7f0f005b app:id/expanded_menu}

org.kiwix.kiwixmobile.tests.DownloadTest > downloadTest[GT-I9300 - 4.3] FAILED 
        android.support.test.espresso.NoMatchingViewException: No views in hierarchy found matching: (with id: org.kiwix.kiwixmobile:id/get_content_card and with text: is "Get Content")
        If the target view is not part of the view hierarchy, you may need to use Espresso.onData to load it from one of the following AdapterViews:android.support.v7.view.menu.ExpandedMenuView{42b71620 VFED.VC. .F...... 0,0-592,387 #7f0f005b app:id/expanded_menu}

org.kiwix.kiwixmobile.tests.NetworkTest > networkTest[GT-I9300 - 4.3] FAILED 
        android.support.test.espresso.NoMatchingViewException: No views in hierarchy found matching: (with id: org.kiwix.kiwixmobile:id/get_content_card and with text: is "Get Content")
        If the target view is not part of the view hierarchy, you may need to use Espresso.onData to load it from one of the following AdapterViews:android.support.v7.view.menu.ExpandedMenuView{426c1d10 VFED.VC. .F...... 0,0-592,387 #7f0f005b app:id/expanded_menu}

org.kiwix.kiwixmobile.tests.ZimTest > zimTest[GT-I9300 - 4.3] FAILED 
        android.support.test.espresso.NoMatchingViewException: No views in hierarchy found matching: WebView with JS enabled
        If the target view is not part of the view hierarchy, you may need to use Espresso.onData to load it from one of the following AdapterViews:android.support.v7.view.menu.ExpandedMenuView{4294e920 VFED.VC. .F...... 0,0-592,872 #7f0f005b app:id/expanded_menu}

org.kiwix.kiwixmobile.utils.BasicTest > basicTest[GT-I9300 - 4.3] FAILED 
        android.support.test.espresso.NoMatchingViewException: No views in hierarchy found matching: (with class name: is "android.widget.ImageButton" and has parent matching: (with id: 2131689583 and has parent matching: with id: 2131689582) and is displayed on the screen to the user)

org.kiwix.kiwixmobile.utils.ContentTest > contentTest[GT-I9300 - 4.3] FAILED 
        java.lang.NoSuchMethodError: org.kiwix.kiwixmobile.testutils.TestUtils.preGrantStorage
        at org.kiwix.kiwixmobile.utils.ContentTest.permissions(ContentTest.java:42)
:app:connectedKiwixDebugAndroidTest FAILED

[mhutti1]

org.kiwix.kiwixmobile.utils.ContentTest has not existed for months. Perhaps try unistalling and reinstalling/making sure you are on the latest branch.

[julianharty]

@mhutti1 On master I can see https://github.com/kiwix/kiwix-android/blob/master/app/src/androidTestKiwix/java/org/kiwix/kiwixmobile/tests/ContentTest.java which was last updated 21 days ago.

Because of the strange errors (calling methods that no longer exist) I'll do a fresh checkout of kiwix-android into a fresh folder in case there's something fishy in my current folder (which was several months out of date before I did a git pull on master.

[mhutti1]

@julianharty Yeah that should be up to date. I really don't know why its erroring with that message.

[julianharty]

An update with some good news. I finally realised that running ./gradlew clean might help. In fact I ran ./gradlew clean cAT (cAT is shorthand for connectedAndroidTest) and that seems to have cleared away the stale copies of the test code. I now have 7 tests that run, 1 database test, and 6 for org.kiwix.kiwixmobile.tests.

These tests have run successfully on many of my devices so far. These include

• Nexus 4 running Android 5.0.1
• Galaxy Nexus running Android 4.3 (yes really this archaic phone :)
• Samsung Galaxy Core Prime running Android 5.1.1
• Nexus 7 (2012) running Android 5.1.1
• HTC Desire 510 running Android 4.4.3

BTW: I had to disable screen animations in the Developer Options for a couple of these phones. You're told to do this when the relevant tests fail on the device & it's a one-off change so not a major issue, see below for an example of the message.

android.support.test.espresso.PerformException: Error performing 'load adapter data' on view 'Animations or transitions are enabled on the target device.
For more info check: http://goo.gl/qVu1yV

Several tests fail on my Samsung GT-I9300 running Android 4.3 as the menu appears near the bottom of the screen which is not drawn properly. I've recorded a video of the tests running which I can provide separately. However I'm going to try and do some more investigation anyway as Kiwix, the app, seems to display the menu correctly so it seems some sort of strange problem emerges when the tests are running.

I'll cover the Samsung S7 next. 3 of the tests still fail because the 'Overlay Detected' message still appears. This is despite me disabling every installed app from being able to draw over other apps.

Starting 7 tests on SM-G930F - 6.0.1

org.kiwix.kiwixmobile.tests.ContentTest > contentTest[SM-G930F - 6.0.1] FAILED 
        android.support.test.espresso.NoActivityResumedException: No activities in stage RESUMED. Did you forget to launch the activity. (test.getActivity() or similar)?
        at dalvik.system.VMStack.getThreadStackTrace(Native Method)

org.kiwix.kiwixmobile.tests.DownloadTest > downloadTest[SM-G930F - 6.0.1] FAILED 
        java.lang.RuntimeException: Could not launch intent Intent { act=android.intent.action.MAIN flg=0x10000000 cmp=org.kiwix.kiwixmobile/.utils.SplashActivity VirtualScreenParam=Params{mDisplayId=-1, null, mFlags=0x00000000)} } within 45 seconds. Perhaps the main thread has not gone idle within a reasonable amount of time? There could be an animation or something constantly repainting the screen. Or the activity is doing network calls on creation? See the threaddump logs. For your reference the last time the event queue was idle before your activity launch request was 1500752744907 and now the last time the queue went idle was: 1500752789926. If these numbers are the same your activity might be hogging the event queue.
        at android.support.test.runner.MonitoringInstrumentation.startActivitySync(MonitoringInstrumentation.java:360)

org.kiwix.kiwixmobile.tests.NetworkTest > networkTest[SM-G930F - 6.0.1] FAILED 
        android.support.test.espresso.NoMatchingViewException: No views in hierarchy found matching: (Child at position 0 in parent with id: org.kiwix.kiwixmobile:id/library_list and is displayed on the screen to the user)

:app:connectedKiwixDebugAndroidTest FAILED

I'll agree with @mhutti1 that the tests should be able to run on this device. BTW I managed to get Kiwix to run independently only once I'd enabled the permission in Settings-...->App Manager->Kiwix

I'll continue to run the tests on several more devices for completeness.

[julianharty]

Tests passed on a Nexus 7 (2013) running Android 4.4.4

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

[kelson42]

@macgills @julianharty What should we do with that old automated test problem?

[macgills]

I am happy to close this