Detox: Tapping broken in latest version

[j320]

Description

I've updated Detox and tapping returns Test Failed: View "_element_" is not hittable at point _coordinates_. It doesn't matter if the element is a _View_, _TextView_ or if it's hitting a _TouchableOpacity_ or anything in particular, the tap always fails.

I can confirm that this doesn't happen on Android

• [ ] I have tested this issue on the latest Detox release and it still reproduces

Environment (please complete the following information):

• Detox: 16.9.2 & 17.0.1
• React Native: 0.62.2
• Node: 14.0.0
• Device: iPhone 11
• Xcode: 11.5
• iOS: 13.5
• macOS:

Logs

https://gist.github.com/j320/67857ba0e3224fd48ece641b2151cbec

[LeoNatan]

Thanks will look next week.

[LeoNatan]

I will release a patch soon with improved visibility/hit testing, and with improved error logging, so I will ask that you try soon and see if it was fixed, or at least if a better error shows up. Thanks

[LeoNatan]

17.0.2 out. Please test.

[todorone]

@LeoNatan Issue is still there with 17.0.2 for me. I'm using TouchableOpacity of react-native-gesture-handler...

Test Failed: View “<RCTView: 0x7f8c168344f0>” is not hittable at point “{"x":70,"y":28}”: Another view “<RNGestureHandlerButton: 0x7f8c168336e0>” is hittable at window point “{"x":160,"y":424}”

[LeoNatan]

@todorone Can you please post the whole hierarchy that comes with that error? If you don’t see a hierarchy, please run your test with --loglevel verbose. Thanks

[todorone]

@LeoNatan Sure, here it is. Thanks.

    Test Failed: View “<RCTView: 0x7ff42d724e40>” is not hittable at point “{"x":70,"y":28}”: Another view “<RNGestureHandlerButton: 0x7ff42d737fa0>” is hittable at window point “{"x":160,"y":424}”
    View Hierarchy:
    <UIWindow: 0x7ff42f007130; frame = (0 0; 320 568); gestureRecognizers = <NSArray: 0x6000018ff870>; layer = <UIWindowLayer: 0x6000010fdb20>>
       | <UITransitionView: 0x7ff42f01e5c0; frame = (0 0; 320 568); autoresize = W+H; layer = <CALayer: 0x6000010fec00>>
       |    | <UIDropShadowView: 0x7ff42d72b480; frame = (0 0; 320 568); clipsToBounds = YES; autoresize = W+H; layer = <CALayer: 0x6000016ae5e0>>
       |    |    | <RCTRootView: 0x7ff42d61af60; frame = (0 0; 320 568); autoresize = W+H; layer = <CALayer: 0x6000016eae60>>
       |    |    |    | <RCTRootContentView: 0x7ff432004d20; reactTag: 1; frame = (0 0; 320 568); gestureRecognizers = <NSArray: 0x6000018aadf0>; layer = <CALayer: 0x6000016ad5c0>>
       |    |    |    |    | <RCTView: 0x7ff42f0261e0; reactTag: 5; frame = (0 0; 320 568); layer = <CALayer: 0x60000101a480>>
       |    |    |    |    |    | <RCTView: 0x7ff42d64a070; reactTag: 3; frame = (0 0; 320 568); layer = <CALayer: 0x600001015be0>>
       |    |    |    |    |    |    | <RCTView: 0x7ff43200fb80; reactTag: 39; frame = (0 0; 320 568); layer = <CALayer: 0x60000101e220>>
       |    |    |    |    |    |    |    | <RCTView: 0x7ff43200f8e0; reactTag: 37; frame = (0 0; 320 568); layer = <CALayer: 0x60000101e1e0>>
       |    |    |    |    |    |    |    |    | <RNCSafeAreaProvider: 0x7ff43200f3f0; reactTag: 35; frame = (0 0; 320 568); layer = <CALayer: 0x60000101e080>>
       |    |    |    |    |    |    |    |    |    | <RCTView: 0x7ff43200eeb0; reactTag: 29; frame = (0 0; 320 568); layer = <CALayer: 0x60000101df20>>
       |    |    |    |    |    |    |    |    |    |    | <RCTView: 0x7ff42d72e2e0; reactTag: 123; frame = (0 0; 320 568); layer = <CALayer: 0x600001027900>>
       |    |    |    |    |    |    |    |    |    |    |    | <RCTView: 0x7ff42f1253c0; reactTag: 43; frame = (0 0; 320 0); layer = <CALayer: 0x600001004820>>
       |    |    |    |    |    |    |    |    |    |    |    | <RCTView: 0x7ff42d72e040; reactTag: 119; frame = (0 0; 320 568); layer = <CALayer: 0x6000010278e0>>
       |    |    |    |    |    |    |    |    |    |    |    |    | <RCTView: 0x7ff42d72dda0; reactTag: 117; frame = (0 0; 320 568); layer = <CALayer: 0x6000010278c0>>
       |    |    |    |    |    |    |    |    |    |    |    |    |    | <RCTView: 0x7ff42d72db00; reactTag: 115; frame = (0 0; 320 568); gestureRecognizers = <NSArray: 0x60000185c870>; layer = <CALayer: 0x600001027880>>
       |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <RCTView: 0x7ff42f0267e0; reactTag: 45; frame = (0 0; 3 568); userInteractionEnabled = NO; layer = <CALayer: 0x6000010338c0>>
       |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <RCTView: 0x7ff42d72b8f0; reactTag: 113; frame = (0 0; 320 568); clipsToBounds = YES; layer = <CALayer: 0x600001027840>>
       |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <RCTView: 0x7ff42d72b650; reactTag: 109; frame = (0 0; 320 568); layer = <CALayer: 0x6000010277e0>>
       |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <RCTView: 0x7ff42d7307d0; reactTag: 107; frame = (0 0; 320 568); layer = <CALayer: 0x600001027800>>
       |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <RCTView: 0x7ff42d72ca00; reactTag: 105; frame = (0 0; 320 568); layer = <CALayer: 0x6000010278a0>>
       |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <RCTView: 0x7ff42f126200; reactTag: 49; frame = (98.5 0; 123 394); layer = <CALayer: 0x6000010049a0>>
       |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <RCTImageView: 0x7ff42f025620; reactTag: 47; frame = (0 175; 123 44); clipsToBounds = YES; layer = <CALayer: 0x600001004840>>
       |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <RCTUIImageViewAnimated: 0x7ff42f125db0; baseClass = UIImageView; frame = (0 0; 123 44); opaque = NO; autoresize = W+H; userInteractionEnabled = NO; layer = <CALayer: 0x6000010048a0>>
       |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <RNGestureHandlerButton: 0x7ff42d737fa0; baseClass = UIControl; frame = (48 394; 224 60); clipsToBounds = YES; layer = <CALayer: 0x6000010275a0>>
       |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <RCTView: 0x7ff42d724e40; reactTag: 57; frame = (42 2; 140 56); layer = <CALayer: 0x600001027560>>
       |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <RCTTextView: 0x7ff42f1266c0; reactTag: 55; text: Get started frame = (14.5 15.5; 111 23.5); text = 'Get started'; opaque = NO; layer = <CALayer: 0x600001004980>>
       |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <RCTTextView: 0x7ff42d730170; reactTag: 89; text: By signing up, you agree to the
    Terms of Service and Privacy Policy.[E2E] frame = (43 499; 234 49); text = 'By signing up, you agree ...'; opaque = NO; layer = <CALayer: 0x6000010276c0>>
       |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <RCTTextView: 0x7ff42d730610; reactTag: 103; text: 0.2.5(0) frame = (270.5 552.5; 48.5 16.5); text = '0.2.5(0)'; opaque = NO; layer = <CALayer: 0x600001027760>>
       |    |    |    |    |    |    |    |    |    | <RCTView: 0x7ff43200f150; reactTag: 33; frame = (0 0; 320 0); layer = <CALayer: 0x60000101e020>>

[LeoNatan]

I think I understand what’s going on.
How do you select this element in your test? You should be selecting the parent, rather than that element. I assume it’s some sort of wrapper, and that middle element (between the text and the gesture handler button) has pointer-events: none;, which makes the view decline touches in the responder system.

If you think about it, the failure makes sense, because you are asking the system to tap on a view that was explicitly defined as untappable. In Detox 17, we are more strict with our interactions, and ask that the correct views be targeted for actions such as taps.

[j320]

Still failing in the last version: https://gist.github.com/j320/54c9bba4742af8226f8c74b623989058

I've tested tapping the element at different levels of the hierarchy but they all fail.

I think that the way it was working before and still does on Android is fine, it allows for less code to be written as you can use the same testID for, for example, expecting text and tapping as well rather than adding more testID and complexity to the test suites.

Anyhow, I still haven't found a way to tap in the last version.

[todorone]

@LeoNatan Yes, if we assign testID to parent, tap is working.

At the same time, I just checked - there is no pointer-events props on either parent or children, so Detox probably do not accept taps on regular View which i think is not good at all. This is e2e tests and sometimes we want to imitate user to tap on some View regardless of it being touchable or not and previous implementation worked in such a way. I would ask to reconsider this behaviour is possible.

Thanks.

[todorone]

@j320 Little adjustment need to be done to react-native-gesture-handler library to make things working...

[LeoNatan]

@todorone Detox can send tap actions to regular views, as long as they accept a touch. In this case, the view rejects the tap.

[todorone]

@j320 That's the PR fixing the issue - https://github.com/software-mansion/react-native-gesture-handler/pull/1104

[LeoNatan]

This is the logic inside RCTView, deciding whether to accept a touch or not:

https://github.com/facebook/react-native/blob/aee88b6843cea63d6aa0b5879ad6ef9da4701846/React/Views/RCTView.m#L309

Something here stops the hit test mechanism, and this fails Detox.

[LeoNatan]

I'm closing the issue here. All issues I'm aware of have been resolved, or will be resolved when the open source imports the PR.

[Guuz]

Still getting the issue... Any ideas on how to make Detox and RNGH compatible? Both are much-used libs.

Test Failed: View “” is not hittable at point “{"x":58.5,"y":12.166664123535156}”: Another view “” is hittable at window point “{"x":187.5,"y":210.16664886474609}”

[LeoNatan]

Can you catch this with Xcode (by following our Xcode integration guide) and look why this button is put in front of your text view.

[Guuz]

@LeoNatan Not sure what you mean. You probably need the view hierarchy? See screenshots below.

What happens is as follows:
We have a from RNGH. Inside of it is a element. We use Detox by.text("button text") to find the button and tap on it. But this finds the textnode, which probably isnt tappable. It should "bubble" up to the button perhaps? Not sure how this works behind the scenes.

[LeoNatan]

This doesn't help me. I need to see the iOS view hierarchy, not RN's interpretation of it.

Follow this:
https://github.com/wix/Detox/blob/master/docs/Guide.DebuggingInXcode.md

In Xcode, add an exception breakpoint. Reproduce your failing test, and it should hit the break point when the above assertion fails ("View is not hittable at point"). Then click on the view hierarchy button in Xcode, look for both the text view and the button that should be above it. If you find them one on top of the other, there is no bug. If you see different results, post a screenshot here.

[Guuz]

Found what you were asking for, i think :) But not sure if this helps.
This element has the "Not Accessibility Element" attribute in Xcode. But this is also the Button that needs to be pressed, i assume.

This is the text that is probably found with by.text("Mocked Data")

Does this help?

[LeoNatan]

Yes, it does. Can you also post a screenshot of the view properties of the text view (right pane in Xcode)?

I have a feeling the text view somehow doesn't want to catch the touch. Perhaps some pointer events definition for the view in RN? Perhaps the error in Detox is misleading in this case, as I don't think I check directly on the queried view if it can be hit at the request point (I didn't think of subclasses overriding hitTest:withEvent:. So I might add a check for that and add a new error string so it's more clear, but you will still not be able to tap on that view if it decides not to accept touch events.

I suggest adding a testID to your button directly, and tapping that.

[Guuz]

@LeoNatan Maybe i found something else! Below are screenshots of all the views, their UI (highlighted in the middle) and the right pane in xcode. The 3th screenshot seems to be a view (not accessible) overlaying the same area as the button?

[LeoNatan]

But it looks like that view is behind the text view wrapper view, so it shouldn’t interfere, unless the text view actively declines touches.

[Guuz]

Hmm ok! makes sense.
@LeoNatan do you have a "conclusion"? Is there an issue with RNGH and Detox which prevents them from working together? Or is this how it is all supposed to work and are by.text() lookups for buttons not possible anymore?
I'm asking this because if you would use a Touchable from React Native, Detox works fine, but if you use a Touchable from RNGH it doesn't work. This could be confusing for users of the libraries.

[LeoNatan]

I don’t know what the difference is, so I can say for sure. Can you please create a small demo project reproducing it? I will look at the native views, and will be able to answer for sure.

[Guuz]

@LeoNatan Ive made a sample with tests (that fail)
https://github.com/Guuz/detox-rngh
Video for people reading this that don't want to run the project yourself but want to see the issue: https://streamable.com/chsqms

I assume you know how to run this on your machine. I think these are all the steps you need:

1. npm install
2. pod install
3. ./node_modules/.bin/detox build --configuration ios
4. ./node_modules/.bin/detox test --configuration ios

As you can see the RN components can be tapped, but the RNGH components can not.

[LeoNatan]

Thanks. Please keep the repo alive, I will look at it soon.

[LeoNatan]

Found the issue:

https://github.com/software-mansion/react-native-gesture-handler/blob/df5de67f96bae5fe5f0100dde320b1f0bc461b51/ios/RNGestureHandlerButton.m#L62

The button class captures touches that should go to the text view, which makes Detox decide the text view is not tappable. The only solution I see right now with the code as is, is for you to put a testID on the button and discover it in that way.

[lumberman]

Still having this issue with TouchableOpacity in iOS.

Test Failed: View “<RCTView: 0x7fd5a31053d0>” is not hittable at point “{"x":187.5,"y":15}”; Another view “<RNGestureHandlerButton: 0x7fd5a3109540>” is hittable at window point “{"x":187.5,"y":352}”

Doesn't have anything custom in TouchableOpacity implementation.

[losikov]

The only solution I see right now with the code as is, is for you to put a testID on the button and discover it in that way.

You don't always, mostly never, have a control over root elements and their testID, unless you create the elements yourself. For example, Drawer.Navigator allows to set label and icon:
export type NavigationDrawerOptions = { title?: string; drawerLabel?: | React.ReactNode | ((props: DrawerLabelProps) => React.ReactNode); drawerIcon?: React.ReactNode | ((props: DrawerIconProps) => React.ReactNode); drawerLockMode?: DrawerLockMode; };
and, the button is created in the lib itself.
It would be nice of you to look at the detox as 'how nice to use' instead of 'how nice it is written' not to force all libraries fails, including the major fundamental ones. Which are impossible to test now with detox.

[LeoNatan]

Wouldn’t it be nice to have “fundamental” libraries properly support testing?

We’ve discussed allowing a more loose mode internally, but it has not been implemented yet.

[losikov]

Wouldn’t it be nice to have “fundamental” libraries properly support testing?

We’ve discussed allowing a more loose mode internally, but it has not been implemented yet.

1. Not in the way to brake all tests with a library update without a prior note. It is a major cons against using detox, but unfortunately there're no alternative. I would move out to more stable and predictable testing framework. I really love the framework, but hate when mess like this happens.
2. It is a big question if for <ButtonRoot><View/><View/></ButtonRoot>, if touches on separate Views should work or not. To me, they must. To me, if ButtonRoot is created by parent library, I don't want to care of its testID, and which hierarchy it has inside.

[LeoNatan]

The change was made in a major version, with release notes and migration guide. What prior note did you expect exactly?

You might not care, but Detox lives in the native world, where development _should_ happen, and there is no way to generically know a connection between different levels of the hierarchy. In your mind, what you say makes sense. But imagine trying to tap a disabled button and some parent view receiving the touch event. That to me is more critical than you, or “fundamental” libraries, having to add a testID to their view, so that the correct view can be targeted for interaction.

[losikov]

Thanks for the constructive feedback.

What prior note did you expect exactly?

Not sure if you are familiar with react-native-testing-library. A few releases before deprecation, you will get warnings for deprecated methods. I understand, the same method won't work for detox, but it would be nice to notify in advance about such kind of changes. Certain blog, etc would perfectly work.

Let me give you a clear example, not related to Navigation.Drawer which doesn't work now, I don't think will work soon. I have a Button, and 80% is hidden under another view. I exactly know, that Text with "X" text is displayed in other 20% which is not hidden. Can I send tap() to the button using Text..? Why not?

[LeoNatan]

You can provide specific coordinates of the tap, which will then pass the hit test. Check out the API for tap.

In the drawer, why cant the framework provide a way to set a test identifier? This is a smell for the framework, sorry.

Detox has used a different underlying framework for years now. A few months ago, we decided to rewrite this part with our own implementation. This was discussed in several issues. First we thought we’d use Apple’s XCUITest framework (which is much more strict and limited), but then decided to implement it ourselves for more control. We’ve already fixed many issues in this new implementation. We don’t have blogs. Our deprecated API remains supported for years (I don’t think we’ve removed a single deprecated API, ever). We just made the new implementation more accurate, and thus stricter. I see no way to “deprecate” the old behavior because we no longer use the older third party.

[LeoNatan]

https://github.com/wix/Detox/blob/master/docs/APIRef.ActionsOnElement.md#tappoint

[losikov]

I'm aware of tapAtPoint. I use it for example, to tap 'header-back' when normal tap() on certain reason doesn't work:
element(by.id('header-back')).atIndex(i).tapAtPoint({x: 10, y: 15})

It doesn't work with tap(), because I suppose something sits over the point it touches.

For my previous example, when tapping Text object on another Button, why should I touch by coordinates, not the Text object? Doesn't it look contr-intuitive to normal programmer expectation? Access by object, not by {x, y}.

For the case <ButtonRoot><View/><View/></ButtonRoot>, I do expect both Views could be touched separately, even if I have an access to ButtonRoot (which I don't have with Drawer.Navigator and won't have soon or ever).

[losikov]

I access the parent button (item in Navigation Drawer), I had to:
await element(by.type('RNGestureHandlerButton').withDescendant(by.id('My Account'))).tap()
To me, it is really weird.

[LeoNatan]

The way the OS framework works, when you touch the screen, it asks window by window, view by view "is this point inside you and do you want it?". When you tap(point) (tapAtPoint() is deprecated but available), I run the same hit test functionality that iOS does, only to see if the view you want is the one taking the touch. If some other view turns out to get the touch (in your case RNGestureHandlerButton), often there is no way to know if it's part of a <ButtonRoot><View/><View/></ButtonRoot> or a <ParentWithGestureHandler><Something><Something><Something><DisabledButton /></Something></Something></Something></ParentWithGestureHandler>. In your case, you are targeting a text label (that to the OS rejects touches) that could be part of a button, but could also be part of some container that might erroneously catch a touch. So I don't see why asking you to be more exact is so bad. When you specify that your view has a descendant, you are narrowing down the search and also providing the exact target with which you'd like to interact. This API is there for a reason. Now, if your "fundamental" framework was properly written, you could have just given the button an identifier and be done with it, but since you cannot, I don't see what else would satisfy you other than a wild west, and I'm not really convinced it's something I'd like to offer.

Internally, we've had push back for strictness, of course, but we already caught abuses of the previous model, where people were doing things that the user would never do, thus having invalid tests. And then, with Detox 17, that approach broke, and at first it was "but it worked until now, why no more?", and as we investigated, it turned out the tests were odd.

If we lived in the native world, instead of this RN crapshoot, we'd have a much more ordered hierarchy, and things would be much more easier to generalize. This is basically what XCUITest does. It only exposes some of the accessibility aspects of views. But with React Native being the mess that it is, button is not really a button (from a native standpoint), Detox has to shoot for lowest common denominator, which is very low with RN. This is the limitation of this technology.

Thinking about it, even with web stuff like "react-native-testing-library", how can it know if the static text you attempted to tap is hittable or not? It's the same problem, unless RN somehow annotates RNGestureHandlerButton as a hittable that swallows its children's touches. I am not familiar with any such mechanism in JS.

Regarding tap and obscured view; Detox, by default with tap(), attempts to tap the accessibility activation point; it's the same that Voice Over activates when a blind user double taps the screen. If you want Detox to behave as you expect, you should fix your app to provide a proper activation point for that button. Not only will it fix your tests, but make your app accessible. By default, the accessibility activation point is the middle of the view.

https://developer.apple.com/documentation/objectivec/nsobject/1615179-accessibilityactivationpoint

[LeoNatan]

It seems like there are different "normal programmer expectation" between native and JS developers. My expectations are for the native view hierarchy traversal and interaction to be sound in a native world. Detox is a native project, which supports native and RN apps. It assumes basic understand of the underlying OS framework and how they operate, on iOS and on Android. So when you say "normal programmer expectation", I cannot really answer your question; we seem to have different expectations and testing methodologies.

[losikov]

You are probably right, from implementation point of view of the library. But from user point of view, where user is someone who uses detox, absolutely not. You clearly understand that the latest version of detox doesn't work with latest react-navigation, because it doesn't provide testID for all root elements. What is the point to create problems for users who is using detox? If user (someone using detox) wants to send a tap to text, just send it. Don't explain hierarchy under it, that it is not a button, or it can't be touched, under another view, etc. Don't explain that is not a right way to do it because you think so. Think as user, who is going to use the testing framework.

UI is often complex, with overlapping/invisible views. And it is easier to hit object X, where X is Text, than 100X ways you described, including forcing everyone rewrite their frameworks. None of them works. Only stubs over detox.
When I need to read a value from the object, I need the exact object. But when I need to touch.... I don't care if I hit button, text or icon on it. To iOS simulator, there's no difference.

Just another example, which doesn't work any more (worked in 17.4.4). I have full-screen view. I have drawer hidden behind right edge of the phone. How can I pull it out? It is invisible, and detox tells me it. in 17.4.4 I did mainView.swipe(left, fast, 0.999), and it worked - it swiped all screen from left to right, starting from the right edge. From 17.4.5 it swipes from the middle of the screen to the left edge. I don't care how to do it properly, I just need swipe from {X1, Y1} to {X2, Y2}.

You think about implementation. Think bigger, from a user's point of view, from a point of view of someone, who is using what you develop. Thank you!
Detox is nice tool, but when it create such problems in 10h tests... And half of it stops working after next update.

[LeoNatan]

As I explained, Detox 17 uses a completely new interaction system, and it might not be fully stable. You can keep using Detox 16, or you can help us debug. What you describe about swiping sounds like a bug, and you should open an issue, and explain in detail what happens.

You do understand that our intention isn't to put roadblocks for developers, yes? We try to encourage good practices. From early on, we made a decision not to support functionality that encourages bad practice. This approach has worked out, as you said, Detox is the most widely used tool for end-to-end.

[LeoNatan]

OK, I've made the discussed change:

https://github.com/wix/Detox/pull/2328

Interaction will now be as loose as possible, only a small 1x1 point visibility check at the interaction point. Let's see how that works out.

[LeoNatan]

17.5.0