Winappdriver: Find element by Accessibility Id with Ruby

[jsbakker]

I am attempting to build out some cucumber scenario bindings to use WinAppDriver in Ruby. While the Ruby sample Calculator code is enough for me to start a session, launch the app, find elements by name and close the app, I cannot find documentation to help me find an element by accessibility Id. E.g. a clickable UI control that has no text/name and must be identified this way.

I can easily write working code to click on my control using the C# API but I am not getting anywhere when trying to do the same in Ruby. While the C# API is great, doing this in Ruby is important because I'd like to use Cucumber for our UWP app, the same way we are using Calabash for our Android app, which shares the same code using Xamarin.

Much appreciated, thanks.

[beatsm]

Until you get an answer for Ruby, have you tried Specflow for gerkhin / Cucumber goodness?

Get Outlook for Androidhttps://aka.ms/ghei36

---

From: Jeffrey Bakker [e-mail]
Sent: Tuesday, September 12, 2017 12:17:37 AM
To: Microsoft/WinAppDriver
Cc: Subscribed
Subject: [Microsoft/WinAppDriver] Find element by Accessibility Id with Ruby (#262)

I am attempting to build out some cucumber scenario bindings to use WinAppDriver in Ruby. While the Ruby sample Calculator code is enough for me to start a session, launch the app, find elements by name and close the app, I cannot find documentation to help me find an element by accessibility Id. E.g. a clickable UI control that has no text/name and must be identified this way.

I can easily write working code to click on my control using the C# API but I am not getting anywhere when trying to do the same in Ruby. While the C# API is great, doing this in Ruby is important because I'd like to use Cucumber for our UWP app, the same way we are using Calabash for our Android app, which shares the same code using Xamarin.

Much appreciated, thanks.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/Microsoft/WinAppDriver/issues/262, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ADQA1zOuNeXwYARECAsWH_iwso-xq4ncks5shb-QgaJpZM4PT5CK.

[jsbakker]

Thanks @beatsm, I have used SpecFlow in the past, and it will be my last-resort fallback. My goal is to stay in Ruby so we can reuse as much of our Calabash Android test framework as possible for the UWP project.

[jsbakker]

I have found a way to work around this, but there are things to take note. The following doesn't work when creating a Selenium remote driver, but it does work when creating an Appuim driver (which returns a Selenium driver).

$Session.find_elements(:accessibility_id, control_id)[0]

So instead of creating the driver like this:

def caps 
{
    platformName: "WINDOWS",
    platform: "WINDOWS",
    deviceName: "WindowsPC",
    app: "f040f23b-de21-4722-834d-7b617ed7fbe3_6fy7p00psw5z2!App"
}
end

# calculator sample uses this method of creating a driver
$Session = Selenium::WebDriver.for(:remote, :url => "http://127.0.0.1:4723/", :desired_capabilities => caps)

You create it with one of the Appium constructors (using the same capabilities):

opts =
{
    caps:
    {
        platformName: "WINDOWS",
        platform: "WINDOWS",
        deviceName: "WindowsPC",
        app: 'f040f23b-de21-4722-834d-7b617ed7fbe3_6fy7p00psw5z2!App'
    },
    appium_lib:
    {
        wait_timeout: 30,
        wait_interval: 1
    }
}

$Session = Appium::Driver.new(opts, false).start_driver

And in addition to using Appium to create the driver, you also need to set the path when starting the winappdriver server or else it won't work:

winappdriver.exe 127.0.0.1 4723/wd/hub

My only guess is that 'remote' drivers are more designed towards web automation, where accessibility IDs aren't used, and possibly using Appium to create the driver, it explicitly identifies it as a desktop app?

[timotiusmargo]

Hello @jsbakker,

Thank you for sharing your findings. Yes accessibility_id is indeed a locator strategy that is only defined in Appium instead of Selenium. This is also the case for the C# counterpart and as a result, the Appium::Driver is what we need to do instead.

As you may have noticed, the Ruby samples we have is not up-to-date and are missing all the enhancements in the C# calculator sample such as calculator Standard mode enforcement and calculator results trimming. These changes were added to add robustness across the latest Windows 10 versions which have slightly different built in calculator.

Would you be interested to make the Ruby sample up-to-date by bringing in those enhancements, syncing the scenarios, and adding a short README.md as you can find in https://github.com/Microsoft/WinAppDriver/blob/master/Samples/C%23/CalculatorTest/README.md?

[jsbakker]

@timotiusmargo I made some time to give a crack at updating the Ruby sample this afternoon. As much as I could, I made the tests to match out the C# counterparts: Addition, Subtraction, Division and Multiplication, including AccessibilityID and the templated tests using Cucumber (it wasn't fair that Ruby gets a single script and C# gets a whole project with test runner).

I wrote scenario bindings for xpath button name and xpath button automation id elements, but they are not working according to WinAppDriver.exe (but I am sure the query looks right). I am wondering if :xpath is broken on Ruby? Therefore these are unused in the sample and documented. There is now a readme markdown file.

I tried pushing my branch but I don't have access rights to this repository.

[khouzam]

Hi @jsbakker,

Thank you and we are looking forward to your contribution.

Please fork the repo and submit a pull request for the change, this is how we can merge your changes with the code.

[mallen47]

@jsbakker - Thanks for your work on getting a working example for Cucumber/Ruby! This has been a big help! With your example I am able to successfully run the tests against the calculator app when I execute them on my Windows10 machine. However, I'm running into problems running the same tests remotely. That is I have set up a Ruby/Cucumber project on my Mac and I'm attempting to execute the tests remotely against the Appium/WinAppDriver server running on my Windows10 VM (also running on my Mac). When I run the command from my Mac terminal:

bundle exec cucumber --tags @standard --format html --out test_report.html --form pretty
I don't see any activity in the Appium console on the Windows VM. However, all the cukes fail with the same error:

unexpected response, code=400, content-type="text/plain"
Unsupported OS/browser/version/device combo: OS: 'WINDOWS', Browser: 'unspecified', Version: 'latest', Device: 'WindowsPC' (Selenium::WebDriver::Error::WebDriverError)

From this I can't tell if there is even any connection established with the Appium server? I'm not sure if this is a problem with my capabilities or maybe with the way I'm starting the Appium server?

Any suggestions would be most welcome!

[jsbakker]

@mallen47 you're welcome. What I would do is troubleshoot the connection between winappdriver running on the Windows VM and the Mac host. From the host can you ping the VM via the ip and port number from which winappdriver is running? If so, the next step would be to run the following interactively inside of an IRB shell:

1. Start the IRB shell: irb
2. Enter the required libs:
   >require 'selenium-webdriver'
   >require 'rubygems'
   >require 'appium_lib'
3. Paste this command into one line and hit enter:

opts =
    {
        caps:
        {
            platformName: "WINDOWS",
            platform: "WINDOWS",
            deviceName: "WindowsPC",
            app: 'your_app_guid'
        },
        appium_lib:
        {
            wait_timeout: 30,
            wait_interval: 1.5,
            host: "your host address"
        }
    }

1. $AppiumDriver = Appium::Driver.new(opts, false)
2. $Session = $AppiumDriver.start_driver

This will either work, or it won't. If it doesn't I would expect to see some output from the winappdriver.exe or Appium console if a connection is present (I prefer to use winappdriver.exe in a cmd window to rule out Appium issues). I'd say create a new issue with the error message, as this is going off-tpoic.