Detox: typeText fails on iOS with Detox `13.3.1`

[PaulMest]

Describe the bug
✅typeText succeeds with Detox 13.1.1, 13.2.0 (old)
⚠️typeText fails with Detox 13.3.1 (new)

To Reproduce

• [X] I have tested this issue on the latest Detox release and it still reproduces

Provide the steps necessary to reproduce the issue. If you are seeing a regression, try to provide the last known version where the issue did not reproduce.

const emailTextBox = element(by.text('Email Address'));
await emailTextBox.typeText(emailAddress);

This is a regression.
13.2.0 worked fine
13.3.0 and 13.3.1 do not work.

If possible, please provide a small demo project that reproduces the issue, or attach a video with the reproduction - this would be very appreciated.

Expected behavior
It should simulate typing into text box.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• Detox: 13.3.1
• React Native: 0.59.10
• Node: 10.16.0
• Device: MacBook Pro -- iPad 6th Generation Simulator
• Xcode: 10.3 (10G8)
• iOS: 12.4
• macOS: Mojave 10.14.6 (18G84)

Device and Verbose Detox Logs
detox test --loglevel trace: e2e.log

[LeoNatan]

Could you describe what you see during the test? Would be best to have a video of the test running. I ask this because there were changes made in the recent version, and the video may assist me with debugging. Thanks

Our test suites contains various typing tests, so it's odd that it breaks for you.

[PaulMest]

In the failure case, it appears to tap and then it fails and refreshes the app.

I can send you a video in a private Dropbox link if you can share with me your email address. Or follow me on Twitter and I'll send you the link in a DM.

[LeoNatan]

You can post on YouTube.

[LeoNatan]

But if you see the tap, that means that it has found an element. I wonder why it's failing... Will you be up for debugging this in Xcode?

[PaulMest]

I can also upload to YouTube... but it's more of having a forum to send a private link.

As for debugging this in Xcode, sure... send me instructions or let me know if we should find a time to screen share. I'm in L.A. (Pacific Time).

[LeoNatan]

Please follow this guide:
https://github.com/wix/Detox/blob/master/docs/Guide.DebuggingInXcode.md

Once finished, open GREYActions+Detox.m and put a break point in _DTXTypeText. Now run your test (best to put .only so that only this test runs). Once the breakpoint hits, in the bottom, there should be a debugger prompt; type po [[UIWindow keyWindow] firstResponder] and paste here the output. Then type po text and paste the output.

[muraliseveneleven]

I see this issue as well , will run thru the debugger and post here

{ Error: An action failed. Please refer to the error trace below.
Exception with Action: {
"Action Name": "Type 'John'",
"Element Matcher": "((!(kindOfClass('RCTScrollView')) && (((kindOfClass('UILabel') || kindOfClass('UITextField') || kindOfClass('UITextView')) && hasText('First Name')) || (kindOfClass('RCTTextView') && an object with accessibilityLabel "First Name"))) || (((kindOfClass('UIView') || respondsToSelector(accessibilityContainer)) && parentThatMatches(kindOfClass('RCTScrollView'))) && ((kindOfClass('UIView') || respondsToSelector(accessibilityContainer)) && parentThatMatches((((kindOfClass('UILabel') || kindOfClass('UITextField') || kindOfClass('UITextView')) && hasText('First Name')) || (kindOfClass('RCTTextView') && an object with accessibilityLabel "First Name"))))))"
}

  Error Trace: [
    {
      "Description":  "Reason for action failure was not provided.",
      "Error Domain":  "com.google.earlgrey.ElementInteractionErrorDomain",
      "Error Code":  "2",
      "File Name":  "GREYElementInteraction.m",
      "Function Name":  "-[GREYElementInteraction performAction:error:]_block_invoke",
      "Line":  "268"
    }
  ]

-[GREYElementInteraction performAction:error:]_block_invoke

[PaulMest]

Cool, just went through those steps...

>> po [[UIWindow keyWindow] firstResponder]

<RCTUITextField: 0x7fee4b88b800; baseClass = UITextField; frame = (0 0; 392 60); text = ''; opaque = NO; autoresize = W+H; gestureRecognizers = <NSArray: 0x600002fc91a0>; layer = <CALayer: 0x600002195020>>

>> po text
[email protected]

I am thinking now it is an issue with the matcher.

When I use element(by.text('Email Address')) in 13.3.1, typeText fails.
But if I put a testID on the text box and then change the matcher to element(by.id('email')), then typeText succeeds in 13.3.1.

[muraliseveneleven]

@PaulMest are you getting the same error trace ?

Error Trace: [
{
"Description": "Reason for action failure was not provided.",
"Error Domain": "com.google.earlgrey.ElementInteractionErrorDomain",
"Error Code": "2",
"File Name": "GREYElementInteraction.m",
"Function Name": "-[GREYElementInteraction performAction:error:]_block_invoke",
"Line": "268"
}
]

[PaulMest]

@muraliseveneleven Yes. Same error.

    Error Trace: [
      {
        "Description":  "Reason for action failure was not provided.",
        "Error Domain":  "com.google.earlgrey.ElementInteractionErrorDomain",
        "Error Code":  "2",
        "File Name":  "GREYElementInteraction.m",
        "Function Name":  "-[GREYElementInteraction performAction:error:]_block_invoke",
        "Line":  "268"
      }
    ]

My work around was to add a testID and use the by.id matcher. But I don't want to have to do that to all of my components.

[LeoNatan]

That’s very odd. In your debug above, it shows that it has chosen a text view.

[LeoNatan]

I think I see the issue.
You tried to by.text(), but clearly it selected the wrong one:

<RCTUITextField: 0x7fee4b88b800; baseClass = UITextField; frame = (0 0; 392 60); text = ''; opaque = NO; autoresize = W+H; gestureRecognizers = <NSArray: 0x600002fc91a0>; layer = <CALayer: 0x600002195020>>

You can see that it has no text. Are you sure such an element really exists? Is the text a placeholder? It could be some bug with placeholder causing issues.

[PaulMest]

Yeah, exactly. I was trying to match on the placeholder text. It says, Email Address as a placeholder when the field is empty and the real email address (e.g. [email protected]) if someone has typed something in.

This is how the component is defined:

<TextField
  autoCorrect={false}
  autoCapitalize="none"
  autoFocus={noUserSaved}
  style={[styles.input.loginInput]}
  placeholderTextColor={theme.col.black}
  keyboardType="email-address"
  defaultValue={username}
  placeholder="Email Address"
  onChangeText={value => {
    this.setState({ username: value });
  }}
/>

Are you saying:
A) This is not a supported scenario and that I should change my code to use by.id or some other matching method?
... OR ...
B) That you will continue to look into this and have it work like it did in 13.2.0 and below?

[LeoNatan]

I don't see why it should not be supported. I'll try to reproduce your use case and see.

Using by.id() is always recommended because it's the most precise.

[ayelenmarie]

Hi! I've encountered the same issue with the same error trace, but on a element with a testID, that successfully finds but is not able to type anything on:

 An action failed. Please refer to the error trace below.
    Exception with Action: {
      "Action Name":  "Type 'Brooklyn
    '",
      "Element Matcher":  "((!(kindOfClass('RCTScrollView')) && (respondsToSelector(accessibilityIdentifier) && accessibilityID('locations_search'))) || (((kindOfClass('UIView') || respondsToSelector(accessibilityContainer)) && parentThatMatches(kindOfClass('RCTScrollView'))) && ((kindOfClass('UIView') || respondsToSelector(accessibilityContainer)) && parentThatMatches((respondsToSelector(accessibilityIdentifier) && accessibilityID('locations_search'))))))"
    }


    Error Trace: [
      {
        "Description":  "Reason for action failure was not provided.",
        "Error Domain":  "com.google.earlgrey.ElementInteractionErrorDomain",
        "Error Code":  "2",
        "File Name":  "GREYElementInteraction.m",
        "Function Name":  "-[GREYElementInteraction performAction:error:]_block_invoke",
        "Line":  "268"
      }
    ]

The test that fails is the following:

await element(by.id('locations_search')).typeText(`${storeByCity}\n`)

We use a helpers.js file to store the strings that tests use, in this case storeByCity.

I'm using Detox 14.0.0.

[LeoNatan]

There is a PR open with improvements to typing. Let’s see if it helps in these cases. Even if not, it will provide better error messages.

In the case of a placeholder, I don’t think it will be supported. When you select the placeholder view, it’s a label, not the enclosing text field, and so you cannot type in there.

Let’s wait for the PR to see.

[PaulMest]

Would it be possible to build a special matcher by.placeholder that works like by.text in 13.2.0 and earlier?

[LeoNatan]

No. Placeholders are not something universal. iOS has their own implementation, RN, I am assuming, has its own. Those implementations are private and can change at any point. Not something we would want in Detox.

If you insist, you can investigate the view hierarchy and create a complex matcher, where you, for example, match an element whose child contains the text you want. But then the burden of supporting it falls on your, and potentially any new iOS or RN version can break it.

As I said above, using by.id() is recommended because it's stable, even if the initial effort to add identifiers is higher.

I could be wrong about this case in particular, which is why I am not closing the issue. Let's wait for the PR to land.

[LeoNatan]

Detox 14.0.2 is out with the proposed changes. Please try it out and let me know if something improves, or post the new error output. Thanks

@PaulMest @ayelenmarie

[PaulMest]

Here is the new error message (14.0.2) when you use by.text()

    An action failed. Please refer to the error trace below.
    Exception with Action: {
      "Action Name":  "Clear text",
      "Element Matcher":  "((!(kindOfClass('RCTScrollView')) && (((kindOfClass('UILabel') || kindOfClass('UITextField') || kindOfClass('UITextView')) && hasText('Email Address')) || (kindOfClass('RCTTextView') && an object with accessibilityLabel "Email Address"))) || (((kindOfClass('UIView') || respondsToSelector(accessibilityContainer)) && parentThatMatches(kindOfClass('RCTScrollView'))) && ((kindOfClass('UIView') || respondsToSelector(accessibilityContainer)) && parentThatMatches((((kindOfClass('UILabel') || kindOfClass('UITextField') || kindOfClass('UITextView')) && hasText('Email Address')) || (kindOfClass('RCTTextView') && an object with accessibilityLabel "Email Address"))))))"
    }


    Error Trace: [
      {
        "Description":  "Failed to make element [E] first responder.",
        "Description Glossary":    {
          "E":  "<UITextFieldLabel:0x7fb090f02a60; AX=Y; AX.label='Email Address'; AX.frame={{580, 251}, {349, 19.5}}; AX.activationPoint={754.5, 260.75}; AX.traits='UIAccessibilityTraitStaticText'; AX.focused='N'; frame={{8, 20}, {349, 19.5}}; alpha=1; UIE=N; text='Email Address'>"
        },
        "Error Domain":  "com.google.earlgrey.ElementInteractionErrorDomain",
        "Error Code":  "2",
        "File Name":  "GREYActions+Detox.m",
        "Function Name":  "UIView *_ensureFirstResponderIfNeeded(UIView *__strong, NSError *__strong *)",
        "Line":  "127"
      }
    ]

[LeoNatan]

Ha, as I expected. UITextFieldLabel is a private internal Apple class that is placed as a subview of UITextField objects. Surprisingly, iOS' UITextView doesn't support placeholders natively, and so any placeholder implementation for text views is custom. (RN decides which class to use based on the multiline property, iirc.)

What I added in 14.0.2 is support for hierarchy traversal in the opposite way; if you tap a view, and a descendant subview of that view becomes first responder ("focused"), it is accepted and text is typed into it. What isn't supported, and I am not sure is possible to safely support in the general case, is if a view is tapped, and an ancestor view becomes the first responder. This is the case you are facing. Even case-specific hacks, such as accepting the case that the immediate superview becomes first responder, I think, is not safe in 100% of cases. I could add a very specific hack for UITextField+UITextFieldLabel because this is a known iOS case, but then if you have a multiline text input in RN, it won't work for that case.

Let me think about it some more. I will leave this issue open.

For now, I have given you two workarounds; add identifiers to your text inputs (preferred), or create a complex matcher which picks the parent of the element with the given text. https://github.com/wix/Detox/blob/master/docs/APIRef.Matchers.md#advanced

[LeoNatan]

@ayelenmarie I think your issue is different than the one discussed in this thread originally, but due to lazy error reporting in the code, got lumped in the same category. Please see if 14.0.2 solved your issue.

[PaulMest]

Thanks @LeoNatan for the quick turnaround and in-depth explanations. I understand if you don't want to support the scenario given the hairiness of the implementation details. My team will proceed with the by.id guidance until we hear otherwise.

[ayelenmarie]

@LeoNatan Thanks for your detailed approach! I ended up workarounding it by also putting a testID on the Input inside, so I ended up tapping the view around it first and then typing on the Input itself. I will try deleting the Input's testID and see if it works with 14.0.2 update. I'll let you know!

[you1anna]

Hey @LeoNatan any updates on this one? It's blocking us since we have a few apps using a mixture of new and legacy code, so by.text() is the preferred approach until we've moved fully to React Native.

[LeoNatan]

What update do you want? My long texts didn't explain the problem and why it cannot be solved properly for the general case?

I've proposed workarounds which you can use with by.text(), such as using complex matchers.

[you1anna]

Ok thank you for your explanations. We will use by.id() for now.

I don't quite understand why this succeeds before v13.3.1, since this would imply it’s fixable?

[LeoNatan]

I changed the entire typing mechanism to an internal one, instead of what we used in the past, which was Earl Grey based, and had many issues. The new system is supposed to be more robust, but that also means that it is more general in design. Again, if you have ideas how to solve the issues I've brought up before, let's discuss. This is why the issue remains open.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.

Thank you for your contributions!

For more information on bots in this reporsitory, read this discussion.

[you1anna]

Should this be stale? Believe it's something we'll want to fix otherwise by.text() will be flakey.

[LeoNatan]

We have a plan to replace the entire matching and action infrastructure for Detox on iOS. I will see what can be done there. Closing for now.