Detox: Cannot find SwiftUI elements running latest detox version

Created on 12 Nov 2020  ·  15Comments  ·  Source: wix/Detox

Description

Tests fail for SwiftUI elements that cannot be found. These same tests pass when run using detox 16.9.2

  • [ x ] I have tested this issue on the latest Detox release and it still reproduces

Reproduction

Tests pass when run using detox 16.9.2

  1. Update detox to latest version
  2. Run the same test and when the element being looked for is SwiftUI the test fails with error
    > "Error: Test Failed: No elements found for “MATCHER(label == “Current Glucose” && traits ⊇ “[staticText]”)"

Expected behavior

The same test should pass when run with the latest version of detox as it does when run with 16.9.2

Screenshots

Screen Shot 2020-11-12 at 3 21 36 PM

Environment (please complete the following information):

  • Detox: 17.11.2
  • Node: v15.0.1
  • Device: iPhone SE (2nd generation)
  • Xcode: Xcode 12.1 (Build version 12A7403)
  • iOS: 14.1
  • macOS: 10.15.7 (19H2)

acceptebug ios
Source
picture jh-bate

All 15 comments

Also just for my own sanity I ran the test and made it fail on both detox latest & 16.9.2 to show the difference in what I see in the printed View Hierarchy:

detox 16.9.2 (passing)

View Hierarchy with Current Glucose 

|  |  |  |  |  |  |  |  |  |  |  |  |--<SwiftUI.ListCoreCellHost:0x7fe43070a9b0; AX=N; AX.frame={{15, 230.5}, {345, 55.5}}; AX.activationPoint={187.5, 258.25}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 134.5}, {345, 55.5}}; opaque; alpha=1; text=''>
  |  |  |  |  |  |  |  |  |  |  |  |  |  |--<_UITableViewCellSeparatorView:0x7fe43056d600; AX=N; AX.frame={{30, 285.5}, {330, 0.5}}; AX.activationPoint={195, 285.75}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{15, 55}, {330, 0.5}}; opaque; alpha=1>
  |  |  |  |  |  |  |  |  |  |  |  |  |  |--<UITableViewCellContentView:0x7fe43070cfb0; AX=N; AX.frame={{15, 230.5}, {345, 55.5}}; AX.activationPoint={187.5, 258.25}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {345, 55.5}}; opaque; alpha=1>
  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |--<_TtGC7SwiftUIP13$7fff2c9bd5d415ListHostingViewGVS_15ModifiedContentGS1_VS_14_ViewList_ViewVVS_17CellForRowVisitor12CellModifier_VS_19DisappearedModifier__:0x7fe4307c2500; AX=N; AX.frame={{15, 230.5}, {345, 55.5}}; AX.activationPoint={187.5, 258.25}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {345, 55.5}}; opaque; alpha=1>
  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |--<_TtCOCV7SwiftUI11DisplayList11ViewUpdater8Platform13CGDrawingView:0x7fe430557180; AX=N; AX.frame={{288, 251.5}, {49, 20.5}}; AX.activationPoint={312.5, 261.75}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{273, 21}, {49, 20.5}}; alpha=1>
  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |--<_TtGC7SwiftUI16PlatformViewHostGVS_P13$7fff2ca0167832PlatformViewRepresentableAdaptorV9LoopKitUI28DismissibleKeyboardTextField__:0x7fe43055b160; AX=N; AX.frame={{241, 241}, {39, 35}}; AX.activationPoint={260.5, 258.5}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{226, 10.5}, {39, 35}}; opaque; alpha=1>
  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |--<UITextField:0x7fe4308c3c00; AX=N; AX.value='---'; AX.frame={{241, 241}, {39, 35}}; AX.activationPoint={260.5, 258.5}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {39, 35}}; alpha=1; text=''>
  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |--<_UITextFieldCanvasView:0x7fe430531ab0; AX=N; AX.frame={{241, 241}, {39, 35}}; AX.activationPoint={260.5, 258.5}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {39, 35}}; alpha=1; UIE=N>
  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |--<UITextFieldLabel:0x7fe43055e860; AX=N; AX.label='---'; AX.frame={{241, 241}, {39, 33.5}}; AX.activationPoint={260.5, 257.75}; AX.traits='UIAccessibilityTraitStaticText'; AX.focused='N'; frame={{0, 0}, {39, 33.5}}; alpha=1; UIE=N; text='---'>
  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |--<UIAccessibilityTextFieldElement:0x6000033a9580; AX=Y; AX.value='---'; AX.frame={{241, 241}, {39, 35}}; AX.activationPoint={260.5, 258.5}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'>
  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |--<_TtC7SwiftUIP33_A34643117F00277B93DEBAB70EC0697122_UIShapeHitTestingView:0x7fe430545e50; AX=N; AX.frame={{233, 237}, {112, 43}}; AX.activationPoint={289, 258.5}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{218, 6.5}, {112, 43}}; opaque; alpha=1>
  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |--<_TtCOCV7SwiftUI11DisplayList11ViewUpdater8Platform13CGDrawingView:0x7fe430553a40; AX=N; AX.frame={{30, 248.5}, {125, 20.5}}; AX.activationPoint={92.5, 258.75}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{15, 18}, {125, 20.5}}; alpha=1>
  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |--<SwiftUI.AccessibilityNode:0x600001798c80; AX=Y; AX.label='mg/dL'; AX.frame={{288, 251.5}, {49, 20.5}}; AX.activationPoint={312.5, 261.75}; AX.traits='UIAccessibilityTraitStaticText'; AX.focused='N'>
  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |--<UITextField:0x7fe4308c3c00; AX=N; AX.value='---'; AX.frame={{241, 241}, {39, 35}}; AX.activationPoint={260.5, 258.5}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {39, 35}}; alpha=1; text=''>
  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |--<_UITextFieldCanvasView:0x7fe430531ab0; AX=N; AX.frame={{241, 241}, {39, 35}}; AX.activationPoint={260.5, 258.5}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {39, 35}}; alpha=1; UIE=N>
  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |--<UITextFieldLabel:0x7fe43055e860; AX=N; AX.label='---'; AX.frame={{241, 241}, {39, 33.5}}; AX.activationPoint={260.5, 257.75}; AX.traits='UIAccessibilityTraitStaticText'; AX.focused='N'; frame={{0, 0}, {39, 33.5}}; alpha=1; UIE=N; text='---'>
  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |--<UIAccessibilityTextFieldElement:0x6000033a9580; AX=Y; AX.value='---'; AX.frame={{241, 241}, {39, 35}}; AX.activationPoint={260.5, 258.5}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'>
  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |--<SwiftUI.AccessibilityNode:0x6000017988c0; AX=Y; AX.label='Current Glucose'; AX.frame={{30, 248.5}, {125, 20.5}}; AX.activationPoint={92.5, 258.75}; AX.traits='UIAccessibilityTraitStaticText'; AX.focused='N'>

detox latest (failing)

View Hierarchy with Current Glucose not showing

 |    |    |    |    |    |    |    |    |    |    |    |    | <SwiftUI.ListCoreCellHost: 0x7f912dff6980; baseClass = UITableViewCell; frame = (0 134.5; 345 55.5); clipsToBounds = YES; autoresize = W; layer = <CALayer: 0x600002425b60>>
   |    |    |    |    |    |    |    |    |    |    |    |    |    | <UITableViewCellContentView: 0x7f912dff6320; frame = (0 0; 345 55.5); gestureRecognizers = <NSArray: 0x600002a6a910>; layer = <CALayer: 0x600002424280>>
   |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <_TtGC7SwiftUIP13$7fff2c9bd5d415ListHostingViewGVS_15ModifiedContentGS1_VS_14_ViewList_ViewVVS_17CellForRowVisitor12CellModifier_VS_19DisappearedModifier__: 0x7f912dff71f0; frame = (0 0; 345 55.5); autoresize = W+H; gestureRecognizers = <NSArray: 0x600002a6a0a0>; layer = <CALayer: 0x600002426d60>>
   |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <_TtCOCV7SwiftUI11DisplayList11ViewUpdater8Platform13CGDrawingView: 0x7f91688681d0; frame = (15 18; 125 20.5); anchorPoint = (0, 0); opaque = NO; autoresizesSubviews = NO; layer = <CALayer: 0x6000022adc00>>
   |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <_TtC7SwiftUIP33_A34643117F00277B93DEBAB70EC0697122_UIShapeHitTestingView: 0x7f9168868360; frame = (218 6.5; 112 43); anchorPoint = (0, 0); autoresizesSubviews = NO; layer = <CALayer: 0x6000022adc40>>
   |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <_TtGC7SwiftUI16PlatformViewHostGVS_P13$7fff2ca0167832PlatformViewRepresentableAdaptorV9LoopKitUI28DismissibleKeyboardTextField__: 0x7f915df542a0; frame = (226 10.5; 39 35); anchorPoint = (0, 0); tintColor = UIExtendedSRGBColorSpace 0.365 0.475 1 1; layer = <CALayer: 0x6000022a8ac0>>
   |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <UITextField: 0x7f915e8d3a00; frame = (0 0; 39 35); text = ''; opaque = NO; gestureRecognizers = <NSArray: 0x600002a63990>; layer = <CALayer: 0x6000022a8460>>
   |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <UITextFieldLabel: 0x7f915df54850; frame = (0 0; 39 33.5); text = '---'; opaque = NO; userInteractionEnabled = NO; layer = <_UILabelLayer: 0x6000007ea3a0>>
   |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <_UITextFieldCanvasView: 0x7f915df51f80; frame = (0 0; 39 35); opaque = NO; userInteractionEnabled = NO; layer = <_UITextTiledLayer: 0x6000016a1d40>>
   |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    | <_TtCOCV7SwiftUI11DisplayList11ViewUpdater8Platform13CGDrawingView: 0x7f916886c080; frame = (273 21; 49 20.5); anchorPoint = (0, 0); opaque = NO; autoresizesSubviews = NO; layer = <CALayer: 0x6000022ae5a0>>
   |    |    |    |    |    |    |    |    |    |    |    |    |    | <_UITableViewCellSeparatorView: 0x7f915dc6af90; frame = (15 55; 330 0.5); layer = <CALayer: 0x6000022b1840>>
jh-bate picture jh-bate on 12 Nov 2020

Hello,
We recently (Detox 17) dropped support for Earl Grey, in favor of our own solution. Out current solution traverses the view hierarchy to match views. SwiftUI's text and image "views" are not rendered as normal views, so Detox does not find them. I will need to investigate how this is working in EG, but not in our solution.
Thanks

LeoNatan picture LeoNatan on 12 Nov 2020
👍1

Just noticed the hierarchy itself is different? 🧐
Most likely some accessibility feature that we haven't enabled in current Detox. Odd.

LeoNatan picture LeoNatan on 12 Nov 2020

Can you post a sample SwiftUI code for your screen? I'd like to recreate it and investigate. Thanks

LeoNatan picture LeoNatan on 12 Nov 2020

@LeoNatan thanks for that. Yeah I assumed it was to do with the change away from Earl Grey.

We will get together a sample for you. Thanks a lot, we love this project!

jh-bate picture jh-bate on 12 Nov 2020

I think I see something. The hierarchy printed in the above example is a custom dump, which includes accessibility objects, which are not views. Currently, we only look at the view hierarchy tree, rather than the accessibility tree. I need to investigate if it would be possible to extend this right now. We assume too much right now.

LeoNatan picture LeoNatan on 12 Nov 2020
👍1

@LeoNatan here is a sample that fails when run with detox latest and passes using 16.9.2. Hope this helps

https://github.com/jh-bate/TestSample

jh-bate picture jh-bate on 13 Nov 2020

Thank you! Please keep the repo alive, I will use it to see if I can come up with a solution.

LeoNatan picture LeoNatan on 14 Nov 2020

No problems, will leave it there as long as you need it - thanks a lot

jh-bate picture jh-bate on 15 Nov 2020

I've created a PR that adds support for accessibility elements. From limited testing, it seems to work as expected but 🤷‍♂️

Once it passes our internal CI (which is a simple RN test), I will create a prerelease version and post here so you can test in your project. Once we verify internally that all is good and it works for you, I will merge.

LeoNatan picture LeoNatan on 16 Nov 2020

Great - thanks for the super quick turn around

jh-bate picture jh-bate on 16 Nov 2020

It will take a while before we can merge it. There is a PR that I want enter first, and then we will test this internally to make sure nothing broke. Once there is a prerelease version, I will post it here.

LeoNatan picture LeoNatan on 16 Nov 2020

@LeoNatan just confirming these updates are working for me.

jh-bate picture jh-bate on 17 Nov 2020

Great, thanks for letting me know!

LeoNatan picture LeoNatan on 17 Nov 2020

Out

https://github.com/wix/Detox/releases/tag/17.12.0

Thanks

LeoNatan picture LeoNatan on 17 Nov 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

carlosalmonte04 picture carlosalmonte04  ·  4Comments
rborn picture rborn  ·  3Comments
kceb picture kceb  ·  4Comments
JB-CHAUVIN picture JB-CHAUVIN  ·  3Comments
mhyousefi picture mhyousefi  ·  3Comments