Detox: Android Matchers Need To Match More Than Just Visible Elements

[RyanThomas73]

Description

Android matchers only match visible elements. This deviates from the expected behavior and from the behavior when running on iOS.

Steps to Reproduce

Setup a sample app that has an element that will be in the view hierarchy but not considered visible. For example a view that is covered by an overlay with partial transparency.

Run the detox test -c <config> command for an android config with an example test that has an expectation for the 'non visible' element to exist.

Expected Behavior: The test passes because the element exists in the view hierarchy.

Actual Behavior: The test fails on android because the matchers in DetoxMatcher.java only select elements with effective visibility VISIBLE.

Detox, Node, Device, Xcode and macOS Versions

• Detox: 10.0.10
• React Native: 0.57.8
• Node: 11.6.0
• Device: Android Emulator
• Xcode: N/A
• macOS: N/A
• Windows: Windows 10 Pro v1703

Device and verbose Detox logs

Skipping full logs as they're not relevant. The specific error message when matching by test id:

Failed: [Error: Error: No views in hierarchy found matching: (with tag value: is "<Test ID>" and view has effective visibility=VISIBLE)

[LeoNatan]

@amitdwix @rotemmiz I think this is a bug.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.
Thank you for your contributions.

[danielgreane]

+1 confirmed, tested with 12.1.3

[rotemmiz]

Hey @RyanThomas73,

I am trying to find an example to the issue you described. The Detox test suite has something similar (a list view with only a few items visible, but all exist in view hierarchy), this test passes on RN57, but fails in 58 and 59 AFAIK (please correct me if I'm wrong @d4vidi).
The fact that you're using RN57 and your test is failing is what confuses me, can you provide more details or example project?

The issue on later versions of RN is related to https://github.com/facebook/react-native/issues/23870 and needs to be addressed in react native itself.

[RyanThomas73]

Hi @rotemmiz

The most common example I have of this that are problematic to me are custom dialog components that use absolute positioning to cover the screen. I'll reach a point in my test where I need to confirm that my view hierarchy is in a certain state (e.g. the login form is in the hierarchy) but all of the views in the screen are completely covered by the absolute positioned views that compose the dialog.

When I have some available time I'll retest using RN 0.59 and Detox v12 to see if I still encounter the same problems. I suspect I will since the detox matcher logic is still coded to only return views with effective visibility of Visible.

https://github.com/wix/Detox/blob/master/detox/android/detox/src/main/java/com/wix/detox/espresso/DetoxMatcher.java

It looks like they were set to search for views with only effective visibility of Visible in this PR:

https://github.com/wix/Detox/commit/b9a832f665606f130a37c7d924c6cfc5ef014e41#diff-3ceb6941bb5fa2cc651630b78f284949

[fancychimp]

@RyanThomas73 is there a workaround you've found in the meantime?

[RyanThomas73]

@fancychimp Sorry, no work around that I've found. We currently have a gap in our test coverage that we left open until this is resolved for Android.

[fancychimp]

Thanks for your reply, same here 😢 Hopefully this will be addressed soon.

[d4vidi]

@RyanThomas73 I'd like to better understand your use case, if that's ok -- in what exact way do you have some of your views invisible, or partly visible? Also, what's the reasoning behind inspecting views that are hidden by a dialog - wouldn't it make sense to first complete the interaction with the dialog and then go back to the origin screen (e.g. login form)?

I believe a picture's worth a thousand words in this case.

[fancychimp]

@d4vidi, I'm not @RyanThomas73 but I'll share my use case:

In our app, we have a screen that contains an input for a phone number. Once that input is tapped, the number keyboard is brought up. The exact step is await element(by.id('phoneNumber')).typeText('111-111-1111');, and it fails on typeText. The interaction cannot be completed here because of the visibility bug.

[d4vidi]

@fancychimp thanks for the input! To be sure I'm not missing something here - when the keyboard pops up, it usually pops up over the phone number input - or alternatively, pushes it away from the visible viewport? (meaning, the screenshot was taken only after you've manually scrolled back up...)

[fancychimp]

@d4vidi it does not. This is a screenshot at the await element(by.id('phoneNumber')).typeText('111-111-1111'); or await element(by.id('phoneNumber')).tap(); step(s), where I have not scrolled. To my knowledge, nothing should be blocking the input at this point.

[d4vidi]

@fancychimp I agree, yet it seems then that your problem is different seeing that the numbers input is fully visible.

[fancychimp]

@d4vidi even if I try to precisely click on the input's coordinates using tapAtPoint I still get this same error. I don't see an obvious workaround.

[losikov]

Using RN 0.62 & react-navigation 5.X
After I do a push/navigate to next screen with stack navigator, all elements on main view or in navigation header located on root screen are still visible: return true to toBeVisible, and false to toBeNotVisible.

_androidx.test.espresso.base.DefaultFailureHandler$AssertionFailedWithCauseError: 'not is displayed on the screen to the user' doesn't match the selected view.
Expected: not is displayed on the screen to the user
Got: "ReactViewGroup{id=769, desc=Main Menu, visibility=VISIBLE, width=134, height=111, has-focus=false, has-focusable=true, has-window-focus=true, is-clickable=true, is-enabled=true, is-focused=false, is-focusable=true, is-layout-requested=false, is-selected=false, layout-params=android.view.ViewGroup$LayoutParams@63dae5f, tag=OpenDrawerAccount, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=18.0, child-count=1}"_

No issues on iOS. In our case it is used only for convenience, for smart navigation between screens. But it would be nice to get it finally fixed.
facebook/react-native#23870 is older than a year. Is it only dependency?

[rikur]

+1

I have a horizontal and paging FlatList and list items that are not visible are considered visible on Android. The toBeNotVisible() works great on iOS however.

Quick note: iOS uses matcherForSufficientlyVisible(>=0.750000) where as Android seems to use isVisible() (at least on the toBeNotVisible() matcher). Maybe a simple switch to isDisplayingAtLeast(75) would fix this? I think it would at least fix my case with FlatList:

Returns a matcher which accepts a view so long as a given percentage of that view's area is not obscured by any parent view and is thus visible to the user.

If not, at least the feature would be on parity on both platforms.

HTH