Webdriveragent: Can not find elements inside Safari View on Xcode 9

[NickAb]

WebDriverAgent does not list any elements inside SFSafariView on Xcode 9 beta 6, both on iOS 11 and iOS 10.3 simulators.
Inspector returns [Other] - RemoteViewBridge with one [Other] child only and not inner elements.

This worked fine on Xcode 8 + iOS 10.3.

I am using latest WebDriverAgent: 16253f77.

Someone else filed related question on StackOverflow https://stackoverflow.com/questions/45982412/appium-ios-11-remoteviewbridge-access-default-app-sdk-like-photo-email

[NickAb]

Reproducible on Xcode 9 GM Seed (9A235) using iOS 11 GM Seed (15A372) or iOS 10.3

[wrathtodd]

The problem is also reproducible using Appium 1.7.0 + WebDriver on Xcode 9.0 (released) + iOS 11 iPhone 7

[shynkevich-alex]

I have the same issue with real devices on iOS 10.
With device logs:

Sep 20 15:21:44 iPhone WebDriverAgentRunner-Runner(WebDriverAgentLib)[843] : Enqueue Failure: Application local.pid.745 is not running /usr/local/lib/node_modules/appium/node_modules/appium-xcuitest-driver/WebDriverAgent/WebDriverAgentRunner/UITestingUITests.m 37 1

[amedvedjev]

same issue with similar to @wrathtodd config. i did research and sound 2 things:
1) although nothing visible we still can find elements :-) . at least mine predicates are working same as with iOS 10.

    @iOSXCUITFindBy(iOSNsPredicate = "type == 'XCUIElementTypeCell' AND name == 'Camera Roll'")
    private List<IOSElement> cameraRollCell;

2) "mobile:tap" does not work on this view any more correctly. although old way tap for appium working - "new TouchAction((MobileDriver) driver).press(element)..."

[wrathtodd]

@amedvedjev Thanks a lot for the hint with predicates! Modifying mine a little bit helped to locate the element as a workaround

[NickAb]

This issue breaks ability to find elements in web views using xpath locators.

driver.find_element(xpath: '//XCUIElementTypeWebView//XCUIElementTypeTextField')

won't work, but chained non xpath based calls will work

driver.find_element(class: 'XCUIElementTypeWebView').find_element('XCUIElementTypeTextField')

[shynkevich-alex]

I have tried find element in iOS 10 in Safari by NSPredicate but I was not able do it.
Also it was strange because I could print elements tree but can't find any element on page.

[amarnathrayudu]

I am facing same issue with xcode 9.

[dereklholder]

I'm having a similar issue, has anyone found a fix?

[marekcirkos]

Duplicate of https://github.com/facebook/WebDriverAgent/issues/639

I think this relates to iOS 11.0 change that does not mark UIViews as accessible by default anymore :/

[rafe-g]

@marekcirkos even if we mark these view's as accessible:

• Setting the accessibilityIdentifier
• Setting isAccessibilityElement == YES

I still cant see the view's elements. For instance, modifying CNContactPickerViewController view accessibility still wont show the elements in the inspector.

Almost sounds like a bug on Apple. Although I doubt they will fix this. I can see these elements are accessible via Xcode's Accessibility Inspector tool

[marekcirkos]

@rafael-chavez I would still raise an issue with Apple. As regular XCUITest developer you would expect to be able to test app with CNContactPickerViewController right?

[pbhogra]

Hi @marekcirkos

Do we have a solution to the issue of dom elements being not visible for safari view in appium inspector?
Ever since I have upgraded to appium 1.7.1 and xcode 9, I am not able to locate ios elements using xpath on ios 11. These locators are working fine on ios 10.3.3

[DogeWowzie]

Hi @pbhogra:

I found a a workaround for running appium tests with Xcode 9 safari view controller:
https://discuss.appium.io/t/how-to-automate-ui-tests-for-safari-view-controller/6551/6?u=doge

[mykola-mokhnach]

@DogeWowzie Try appium@beta and see if webview elements are visible with Xcode9 SDK there. This PR should do the trick for Facebook WebDriverAgent as well.

[DogeWowzie]

@mykola-mokhnach Thanks! I will give it a try!

[rcuyugan]

Having same issue. Each time I ran test I get an error saying it cannot locate an element with the xpath expression. I've been trying to solve this for days now with no success. Any help would be highly appreciated.

Error: OpenQA.Selenium.InvalidSelectorException: 'Unable to locate an element with the xpath expression //XCUIElementTypeButton[@name="Toggle navigation"])[1] because of the following error:
Error: INVALID_EXPRESSION_ERR: DOM XPath Exception 51'

Setup:
Appium 1.3.1 server 1.72
Mac OS High Sierra
XCode 9.2
iPad mini IOS 9.3.5

Attached screenshots shows, that there is no problem when I am using the inspector. However, when I run the test script, using the same locator found and recorded by the inspector, I get the error. I haven't had any success locating any elements on this setup.

BTW, tried it with different xpaths no luck. Also with accessibility id, it errors with FindbyAccessibilityID is not supported.

Error when running script:
Error: OpenQA.Selenium.InvalidSelectorException: 'Unable to locate an element with the xpath expression //XCUIElementTypeButton[@name="Toggle navigation"])[1] because of the following error:
Error: INVALID_EXPRESSION_ERR: DOM XPath Exception 51'

Script using C#:
public void TestMethod1()
{
MobileDriver.iOSCapabilities();
var el = MobileDriver.iOSInstance.FindElementByXPath("//XCUIElementTypeButton[@name=\"Toggle navigation\"])[1]");
el.Tap(1,1);
}

As you can see. using the same locator captured and recorded by inspector.

Appium server logs:
[debug] [BaseDriver] Event 'initialWebviewNavigated' logged at 1520024365873 (12:59:25 GMT-0800 (PST))
[info] [Appium] New XCUITestDriver session created successfully, session 1dbeefa6-7026-44d7-88f9-7d8c319a9af9 added to master session list
[debug] [BaseDriver] Event 'newSessionStarted' logged at 1520024365874 (12:59:25 GMT-0800 (PST))
[debug] [MJSONWP] Responding to client with driver.createSession() result: {"webStorageEnabled":false,"locationContextEnabled":false,"browserName":"Safari","platform":"MAC","javascriptEnabled":true,"databaseEnabled":false,"takesScreenshot":true,"networkConnectionEnabled":false,"device":"iPad Mini","deviceName":"iPad Mini","platformName":"iOS","platformVersion":"9.3.5","udid":"4936fd622c1f0331270724685596c39a4770d049","automationName":"XCUITest","xCodeOrgId":"removed","CodeSigningId":"iPhone Developer","autoWebview":true,"startIWDP":true,"safariInitialUrl":"https://www.walmart.com/mobile","bundleId":"com.rc.SafariLauncher","showXcodeLog":false,"--session-override":true,"newCommandTimeout":300}
[info] [HTTP] <-- POST /wd/hub/session 200 71578 ms - 701
[info] [HTTP] --> POST /wd/hub/session/1dbeefa6-7026-44d7-88f9-7d8c319a9af9/element {"using":"xpath","value":"//XCUIElementTypeButton[@name=\"Toggle navigation\"])[1]"}
[debug] [MJSONWP] Calling AppiumDriver.findElement() with args: ["xpath","//XCUIElementTypeButton[@name=\"Toggle navigation\"])[1]","1dbeefa6-7026-44d7-88f9-7d8c319a9af9"]
[debug] [XCUITest] Executing command 'findElement'
[debug] [BaseDriver] Valid locator strategies for this request: xpath, id, name, class name, -ios predicate string, -ios class chain, accessibility id
[debug] [BaseDriver] Waiting up to 0 ms for condition
[debug] [RemoteDebugger] Executing 'find_element' atom in default context
[debug] [RemoteDebugger] Sending javascript command (function(){return function(){var k=this;functi...
[debug] [RemoteDebugger] Sending WebKit data: {"method":"Runtime.evaluate","params":{"objectGroup":"console","includeCommandLineAPI":true,"doNotPauseOnExceptionsAndMuteConsole":true,"expression":"(function(){return function(){var k=this;functi...
[debug] [RemoteDebugger] Webkit response timeout: 5000[debug] [RemoteDebugger] Receiving WebKit data: '{"result":{"result":{"type":"string","value":"{\"status\":32,\"value\":{\"message\":\"Unable to locate an element with the xpath expression //XCUIElementTypeButton[@name=\\"Toggle navigation\\"])...'
[debug] [RemoteDebugger] Found handler for message '1'
[info] [HTTP] <-- POST /wd/hub/session/1dbeefa6-7026-44d7-88f9-7d8c319a9af9/element 500 3031 ms - 284 [debug] [XCUITest] Connection to WDA timed out
[debug] [iProxy] recv failed: Operation not permitted[info] [HTTP] --> POST /wd/hub/session {"desiredCapabilities":{"device":"iPad Mini","deviceName":"iPad Mini","platformName":"iOS","platformVersion":"9.3.5","udid":"4936fd622c1f0331270724685596c39a4770d049","automationName":"XCUITest","startIWDP":true,"browserName":"Safari","bundleId":"com.rc.SafariLauncher","showXcodeLog":false,"--session-override":true,"newCommandTimeout":300}}
[debug] [MJSONWP] Calling AppiumDriver.createSession() with args: [{"device":"iPad Mini","deviceName":"iPad Mini","platformName":"iOS","platformVersion":"9.3.5","udid":"4936fd622c1f0331270724685596c39a4770d049","automationName":"XCUITest","xCodeOrgId":"removed","CodeSigningId":"iPhone Developer","autoWebview":true,"startIWDP":true,"browserName":"Safari","bundleId":"com.rc.SafariLauncher","showXcodeLog":false,"--session-override":true,"newCommandTimeout":300},null,null]
[debug] [BaseDriver] Event 'newSessionRequested' logged at 1520024440949 (13:00:40 GMT-0800 (PST))
[info] [Appium] Creating new XCUITestDriver (v2.70.0) session
[info] [Appium] Capabilities:
[info] [Appium] device: iPad Mini
[info] [Appium] deviceName: iPad Mini
[info] [Appium] platformName: iOS
[info] [Appium] platformVersion: 9.3.5
[info] [Appium] udid: 4936fd622c1f0331270724685596c39a4770d049
[info] [Appium] automationName: XCUITest
[info] [Appium] startIWDP: true
[info] [Appium] browserName: Safari
[info] [Appium] bundleId: com.rc.SafariLauncher
[info] [Appium] showXcodeLog: false
[info] [Appium] --session-override: true
[info] [Appium] newCommandTimeout: 300
[info] [Appium] Session override is on. Deleting other 2 active session.
[info] [Appium] Removing session 7e9bba80-a25f-4112-9950-881b9d1b2e32 from our master session list
[info] [Appium] Removing session 1dbeefa6-7026-44d7-88f9-7d8c319a9af9 from our master session list
[debug] [JSONWP Proxy] Proxying [DELETE /session/7e9bba80-a25f-4112-9950-881b9d1b2e32] to [DELETE http://localhost:8100/session/FA32215E-D4D0-4AE1-A4C8-FE791F097B8D] with no body
[debug] [JSONWP Proxy] Got response with status 200: "{\n \"sessionId\" : \"3D59BBDA-2CDC-494D-BC39-FDFA511B3ED5\",\n \"value\" : \"Session does not exist\",\n \"status\" : 6\n}"
[debug] [XCUITest] Unable to DELETE session on WDA: 'A session is either terminated or not started'. Continuing shutdown.
[debug] [XCUITest] Not clearing log files. Use clearSystemFiles capability to turn on.
[debug] [JSONWP Proxy] Proxying [DELETE /session/1dbeefa6-7026-44d7-88f9-7d8c319a9af9] to [DELETE http://localhost:8100/session/3D59BBDA-2CDC-494D-BC39-FDFA511B3ED5] with no body
[debug] [XCUITest] In a web session. Removing remote debugger
[debug] [RemoteDebugger] Disconnecting from WebKit remote debugger
[debug] [RemoteDebugger] WebKit remote debugger socket disconnected[debug] [JSONWP Proxy] Got response with status 200: "{\n \"sessionId\" : \"C2AC9EAF-31EE-4087-B5BE-57CE4C70DC68\",\n \"value\" : {\n\n },\n \"status\" : 0\n}"
[debug] [XCUITest] Not clearing log files. Use clearSystemFiles capability to turn on.
[debug] [XCUITest] In a web session. Removing remote debugger
[debug] [RemoteDebugger] Disconnecting from WebKit remote debugger
[debug] [BaseDriver]
[debug] [BaseDriver] W3C capabilities [] and MJSONWP desired capabilities were provided
[warn] [BaseDriver] Expected W3C "capabilities" to be a JSON Object but was provided with: []
[warn] [BaseDriver] Falling back to MJSONWP desired capabilities
[warn] [BaseDriver] The following capabilities were provided, but are not recognized by appium: device, xCodeOrgId, CodeSigningId, --session-override.
[info] [BaseDriver] Session created with session id: ee7701bf-4250-4e20-b423-0390af1725ed
[debug] [RemoteDebugger] WebKit remote debugger socket disconnected
[debug] [XCUITest] Current user: 'dad'
[debug] [BaseDriver] Event 'xcodeDetailsRetrieved' logged at 1520024441509 (13:00:41 GMT-0800 (PST))[debug] [XCUITest] Available devices: 4936fd622c1f0331270724685596c39a4770d049
[debug] [XCUITest] Creating iDevice object with udid '4936fd622c1f0331270724685596c39a4770d049'
[info] [XCUITest] Determining device to run tests on: udid: '4936fd622c1f0331270724685596c39a4770d049', real device: true[debug] [XCUITest] Xcode version set to '9.2' (tools v9.2.0.0.1.1510905681)
[debug] [XCUITest] iOS SDK Version set to '11.2'
[info] [XCUITest] Safari test requested
[debug] [BaseDriver] Event 'appConfigured' logged at 1520024444070 (13:00:44 GMT-0800 (PST))
[debug] [BaseDriver] Event 'resetStarted' logged at 1520024444071 (13:00:44 GMT-0800 (PST))
[debug] [XCUITest] Reset: running ios real device reset flow
[debug] [BaseDriver] Event 'resetComplete' logged at 1520024444072 (13:00:44 GMT-0800 (PST))
[debug] [iOSLog] Attempting iOS device log capture via libimobiledevice idevicesyslog
[debug] [iOSLog] Starting iOS device log capture with: 'idevicesyslog'[debug] [XCUITest] Crash reports root '/Users/dad/Library/Logs/CrashReporter/MobileDevice/Rob's iPad' does not exist. Got nothing to gather.
[debug] [BaseDriver] Event 'logCaptureStarted' logged at 1520024444481 (13:00:44 GMT-0800 (PST))
[info] [XCUITest] Setting up real device
[info] [XCUITest] Using WDA path: '/Applications/Appium.app/Contents/Resources/app/node_modules/appium/node_modules/appium-xcuitest-driver/WebDriverAgent'
[info] [XCUITest] Using WDA agent: '/Applications/Appium.app/Contents/Resources/app/node_modules/appium/node_modules/appium-xcuitest-driver/WebDriverAgent/WebDriverAgent.xcodeproj'
[debug] [XCUITest] No obsolete cached processes from previous WDA sessions listening on port 8100 have been found
[debug] [JSONWP Proxy] Proxying [GET /status] to [GET http://localhost:8100/status] with no body
[debug] [JSONWP Proxy] Got response with status 200: "{\n \"sessionId\" : \"C2AC9EAF-31EE-4087-B5BE-57CE4C70DC68\",\n \"value\" : {\n \"state\" : \"success\",\n \"os\" : {\n \"sdkVersion\" : \"11.2\",\n \"name\" : \"iPhone OS\",\n \"version\" : \"9.3.5\"\n },\n \"ios\" : {\n \"ip\" : \"192.168.1.18\",\n \"simulatorVersion\" : \"9.3.5\"\n },\n \"build\" : {\n \"time\" : \"Feb 27 2018 12:57:13\"\n }\n },\n \"status\" : 0\n}"
[info] [XCUITest] Will reuse previously cached WDA instance at 'http://localhost:8100/'. Set the wdaLocalPort capability to a value different from 8100 if this is an undesired behavior.
[debug] [BaseDriver] Event 'wdaStartAttempted' logged at 1520024444585 (13:00:44 GMT-0800 (PST))
[info] [XCUITest] Using provided WebdriverAgent at 'http://localhost:8100/'
[debug] [BaseDriver] Event 'wdaSessionAttempted' logged at 1520024444587 (13:00:44 GMT-0800 (PST))
[debug] [XCUITest] Sending createSession command to WDA
[debug] [JSONWP Proxy] Proxying [GET /status] to [GET http://localhost:8100/status] with no body
[debug] [JSONWP Proxy] Got response with status 200: "{\n \"sessionId\" : \"C2AC9EAF-31EE-4087-B5BE-57CE4C70DC68\",\n \"value\" : {\n \"state\" : \"success\",\n \"os\" : {\n \"sdkVersion\" : \"11.2\",\n \"name\" : \"iPhone OS\",\n \"version\" : \"9.3.5\"\n },\n \"ios\" : {\n \"ip\" : \"192.168.1.18\",\n \"simulatorVersion\" : \"9.3.5\"\n },\n \"build\" : {\n \"time\" : \"Feb 27 2018 12:57:13\"\n }\n },\n \"status\" : 0\n}"
[info] [XCUITest] Forcing shouldUseSingletonTestManager capability value to true, because of known XCTest issues under 9.3 platform version
[debug] [JSONWP Proxy] Proxying [POST /session] to [POST http://localhost:8100/session] with body: {"desiredCapabilities":{"bundleId":"com.apple.mobilesafari","arguments":["-u","http://appium.io"],"environment":{},"shouldWaitForQuiescence":true,"shouldUseTestManagerForVisibilityDetection":true,"maxTypingFrequency":60,"shouldUseSingletonTestManager":true}}[debug] [JSONWP Proxy] Got response with status 200: {"sessionId":"A71BAD4E-2D41-444C-9228-8CC067FC4717","value":{"sessionId":"A71BAD4E-2D41-444C-9228-8CC067FC4717","capabilities":{"browserName":"Safari","device":"ipad","CFBundleIdentifier":"com.apple.mobilesafari","sdkVersion":"9.3.5"}},"status":0}
[debug] [BaseDriver] Event 'wdaSessionStarted' logged at 1520024450315 (13:00:50 GMT-0800 (PST))
[debug] [BaseDriver] Event 'wdaStarted' logged at 1520024450315 (13:00:50 GMT-0800 (PST))
[info] [XCUITest] Skipping setting of the initial display orientation. Set the "orientation" capability to either "LANDSCAPE" or "PORTRAIT", if this is an undesired behavior.
[debug] [BaseDriver] Event 'orientationSet' logged at 1520024450320 (13:00:50 GMT-0800 (PST))
[debug] [BaseDriver] Event 'iwdpStarting' logged at 1520024450320 (13:00:50 GMT-0800 (PST))
[debug] [iOS] Starting ios_webkit_debug_proxy at port 27753 on device 4936fd622c1f0331270724685596c39a4770d049
[debug] [BaseDriver] Event 'iwdpStarted' logged at 1520024450336 (13:00:50 GMT-0800 (PST))
[debug] [XCUITest] Started ios_webkit_debug proxy server at: http://localhost:27753
[debug] [XCUITest] Waiting for initial webview
[debug] [iOS] Waiting for 3000 ms before navigating to view.
[error] [IWDP] [ 'Unable to bind 4936fd622c1f0331270724685596c39a4770d049 on port 27753-27753' ][debug] [iOS] Navigating to most recently opened webview
[debug] [iOS] Retrieving contexts and views
[debug] [iOS] Selecting by url: true (expected url: 'http://appium.io')
[debug] [RemoteDebugger] Getting WebKitRemoteDebugger pageArray: localhost, 27753
[debug] [RemoteDebugger] Sending request to: http://localhost:27753/json
[debug] [RemoteDebugger] Page element JSON: [{"devtoolsFrontendUrl":"/devtools/devtools.html?ws=localhost:27753/devtools/page/10","faviconUrl":"","thumbnailUrl":"/thumb/http://appium.io/","title":"Appium: Mobile App Automation Made Awesome.","url":"http://appium.io/","webSocketDebuggerUrl":"ws://localhost:27753/devtools/page/10","appId":"PID:732"}]
[debug] [iOS] Retrieving contexts and views
[debug] [iOS] Selecting by url: true (expected url: 'http://appium.io')
[debug] [RemoteDebugger] Getting WebKitRemoteDebugger pageArray: localhost, 27753
[debug] [RemoteDebugger] Sending request to: http://localhost:27753/json
[debug] [RemoteDebugger] Page element JSON: [{"devtoolsFrontendUrl":"/devtools/devtools.html?ws=localhost:27753/devtools/page/10","faviconUrl":"","thumbnailUrl":"/thumb/http://appium.io/","title":"Appium: Mobile App Automation Made Awesome.","url":"http://appium.io/","webSocketDebuggerUrl":"ws://localhost:27753/devtools/page/10","appId":"PID:732"}]
[debug] [iOS] Picking webview 'WEBVIEW_10'
[debug] [iOS] Attempting to set context to 'WEBVIEW_10'
[debug] [RemoteDebugger] WebKit debugger web socket connected to url: ws://localhost:27753/devtools/page/10
[debug] [RemoteDebugger] Unregistering from page readiness notifications
[debug] [BaseDriver] Event 'initialWebviewNavigated' logged at 1520024453373 (13:00:53 GMT-0800 (PST))
[info] [Appium] New XCUITestDriver session created successfully, session ee7701bf-4250-4e20-b423-0390af1725ed added to master session list
[debug] [BaseDriver] Event 'newSessionStarted' logged at 1520024453375 (13:00:53 GMT-0800 (PST))
[debug] [MJSONWP] Responding to client with driver.createSession() result: {"webStorageEnabled":false,"locationContextEnabled":false,"browserName":"Safari","platform":"MAC","javascriptEnabled":true,"databaseEnabled":false,"takesScreenshot":true,"networkConnectionEnabled":false,"device":"iPad Mini","deviceName":"iPad Mini","platformName":"iOS","platformVersion":"9.3.5","udid":"4936fd622c1f0331270724685596c39a4770d049","automationName":"XCUITest","startIWDP":true,"bundleId":"com.rc.SafariLauncher","showXcodeLog":false,"--session-override":true,"newCommandTimeout":300}
[info] [HTTP] <-- POST /wd/hub/session 200 12430 ms - 649
[info] [HTTP] --> POST /wd/hub/session/ee7701bf-4250-4e20-b423-0390af1725ed/element {"using":"xpath","value":"//XCUIElementTypeButton[@name=\"Toggle navigation\"])[1]"}
[debug] [MJSONWP] Calling AppiumDriver.findElement() with args: ["xpath","//XCUIElementTypeButton[@name=\"Toggle navigation\"])[1]","ee7701bf-4250-4e20-b423-0390af1725ed"]
[debug] [XCUITest] Executing command 'findElement'
[debug] [BaseDriver] Valid locator strategies for this request: xpath, id, name, class name, -ios predicate string, -ios class chain, accessibility id
[debug] [BaseDriver] Waiting up to 0 ms for condition
[debug] [RemoteDebugger] Executing 'find_element' atom in default context
[debug] [RemoteDebugger] Sending javascript command (function(){return function(){var k=this;functi...
[debug] [RemoteDebugger] Sending WebKit data: {"method":"Runtime.evaluate","params":{"objectGroup":"console","includeCommandLineAPI":true,"doNotPauseOnExceptionsAndMuteConsole":true,"expression":"(function(){return function(){var k=this;functi...
[debug] [RemoteDebugger] Webkit response timeout: 5000[debug] [RemoteDebugger] Receiving WebKit data: '{"result":{"result":{"type":"string","value":"{\"status\":32,\"value\":{\"message\":\"Unable to locate an element with the xpath expression //XCUIElementTypeButton[@name=\\"Toggle navigation\\"])...'
[debug] [RemoteDebugger] Found handler for message '1'
[info] [HTTP] <-- POST /wd/hub/session/ee7701bf-4250-4e20-b423-0390af1725ed/element 500 705 ms - 284 [debug] [XCUITest] Connection to WDA timed out
[debug] [iProxy] recv failed: Operation not permitted