Winappdriver: State of WinAppDriver

[kat-y]

Hi everyone, I (@kat-y) am the new Program Manager for WinAppDriver, so please feel free to direct your questions and concerns my way. I’d like to address the limbo and uncertainty around WinAppDriver – much of this is due to an internal transition of WinAppDriver ownership within the company, where it recently moved closer to the team that owns and supports our modern native Windows UI frameworks. This transition is complete, and WinAppDriver now lives with the same team that owns WinUI, WPF, React Native for Windows + macOS, and other products such as the Windows Terminal.

Presently, the team’s focus is on these aforementioned UI Frameworks. This means that, for the next ~6+ months, WinAppDriver development will remain paused. In the interim, we’ll be working in the repo with the community to construct an execution plan for when we do bring dev resources back to WinAppDriver.

Also, as I’ve been ramping up, I’ve taken notice at the many requests for WinAppDriver to be open-sourced. Open sourcing is a lot like any feature we’d consider adding to WinAppDriver: it takes resources to do it since we have to ensure the code is free of private APIs, represents modern OSS standards for Microsoft, has processes & license agreements properly reviewed by our legal team, etc. So ‘going open source’ is definitely on the list of things for the future of WinAppDriver, but this will also have to wait until we can apply more resources to the project in general. I’m looking forward to when we’ll be able to do that.

If you have any questions about what I wrote above, or about WinAppDriver in general, please reach out and ask. Thanks, I’m excited to have a chance to work with this great community!

[444ghost]

Hello,

I hope you do your best to bring back the dev resources to WinAppDriver asap. It seems like there's a memory-leak-related bug in WinAppDriver which I think it's very critical, and it has been around for almost 2 years but has not been knocked out.

https://github.com/microsoft/WinAppDriver/issues/991

Thanks.

[oren451]

Hi @kat-y!
Joining other users hopes WAD returns to be under dev.
A lot of product refinements need to go through to be the next thing, here some examples:

• Improve by far WAD finding element algorithms speed
• Many bugs, some critical to fix
• Create sessions over HWND of not top-level screens
• Extend the supported API
• Insert UI Automation API to WAD API (Instead using getAttribute('LegacyIAccessible.State')
• WinAppDriver need to be open-sourced
• Supporting also Java UI technologies
• Supporting ActiveX elements

Thanks and Good Luck!

[Daniwin82]

thanks for the update

[anunay1]

I have around 200 test cases implemented with WAD, but off late I am started observing after running for 3 to 4 hours the send keys API starts behaving weirdly, the API is not disposed properly even after driver.quit. its starts sending weird characters.

[anunay1]

Also it will be good to have logs with time stamp as the console logs are not very helpful.

[Wolfe1]

Thanks for the update, hope to see resources here soon!

I would suggest running through the open pull requests and merging the ones that still make sense to start cleaning up this github. The issues tab will be its own beast :worried:

[mattboroi]

@kat-y, looking for information regarding the publicly available Microsoft.WinAppDriver.Appium.WebDriver NuGet package.

• What are the similarities and differences between the Microsoft.WinAppDriver.Appium.WebDriver NuGet package and WinAppDriver released on GitHub?
• Are there plans for a WinAppDriver NuGet package that reflects the latest release on GitHub?

I did attempt to research open/closed GitHub issues and the Wiki/FAQ although I had not come across information regarding when or how to use the NuGet package verse installing the GitHub release.

Any information would be greatly appreciated and I'm very excited about all the recent activity behind this project and what it will produce for the community.

[varun210619]

Hi @kat-y
Really good to see WinAppDriver is up and running. Hope there will be many enhancements to the tool in near future.

Could you please look into below issues-

https://github.com/microsoft/WinAppDriver/issues/507
I work in Nordic company and we work on Danish, Swedish, Finnish keyboards.
When I send keys with value XXX-XX, winappdriver prints XXX+XX. Although there are many workarounds for typing correct required values, but please see if you can priotize it.

https://github.com/microsoft/WinAppDriver/issues/1357
My automation scripts contains many steps and sometimes I am required to use xpath.
To validate any relative xpath, i have to execute whole script till the step where xpath is used. This consumes lot of time.
Is it possible to create additional utility on top of winappdriver, where i just enter my xpath and tool highlights that element or provide an error.
I am looking for utility similar console provided in chrome where i can enter xpath and css to validate in web application.

https://github.com/microsoft/WinAppDriver/issues/1378
I am finding winappdriver performance very slow. It takes a long time in step execution where xpath is used or where any table/grids appear.
Even with tree views, it is very slow in execution.

https://github.com/microsoft/WinAppDriver/issues/1361
LegacyAccessible.Value attribute not working in winappdriver. No LegacyAccessible properties work with winappdriver.
Sometimes, this is the only property to be used uniquely, but due to this issue, it becomes a bottleneck.

https://github.com/microsoft/WinAppDriver/issues/975
Fortunately i was able to get the developer settings enabled in my company, but it could be a show stopper or delay in getting these settings.

https://github.com/microsoft/WinAppDriver/issues/1103
You may close this one since WinAppDriver is up and alive :)

More enhancements but i have not opened any issues for these -
I dont want to launch winappdriver.exe manually or even in the code. Is it possible that WindowsDriver itself deals with this whenever WinDowsDriver object is created.

[HenningL]

Highest priority for me would be the issue with processId not found. It seems for me that this is a general windows problem which might get compensated whith some internal knowledge. I recognized for some application (e.g. calc.exe) that even if i start the ".exe" programmatically and retrive the pid , when i insepect or watch tasklist, this pid doesnt even exist anywhere. The binding pid to session is the biggest hurdle to even start with WAD.

[liljohnak]

Would it be possible to implement scroll logic?

[rameshrc]

why do we need to install WinAppDriver.exe?

[riverar]

I think first and foremost, the README.md should reflect that new management has taken over and point to the new, supported feedback channels. I, and I'm sure other windev mvps, would love to help mop up too. (Feel free to reach out to the Windows Development alias; @DHowett @crutkas and others can help route.)

[fforjan]

In our case, we are still doing desktop application, engineering type, meaning we are testing that the application can test a workflow for 3 to 8 hours in a row. (Same as if you were testing Visual Studio)
We have concern that Appium is having issues like memory leaks and will not be able to manage such use cases.

I would be also interested to see a white paper/blog post nowadays for usage of Appium in complex application like Visual Studio or Word and its integration into CI (CD?) workflows

[daniel-bsci]

We are looking into using WAD for our testing needs of WPF applications. But the production environment is heavily regulated and being forced to enable developer mode on system under test might be a deal breaker. Seems it should not be needed to automate WPF apps so removing that requirement is high on our list.

[liljohnak]

@daniel-bsci we are using it just fine with your specs. Also using chromedriver to test our chef embedded controls. Product does not ship with developer mode, just tested. Is there a benefit to your request?

[daniel-bsci]

@liljohnak I updated my post to clarify. Our devices are highly regulated, which includes changes to the OS. Not sure our QA would buy performing formal testing on a device with developer mode on when it won't be enabled in production.

[jayaprakashkottapalli]

Hi Kat-y,
Congrats on your new role.

we are looking for a fix for an issue where running the appium for updating the value in the combobox/editbox available in a cell on windows application using infragistic wpf(19.1.20191.208) grid control takes lot of time locating the control within the cell.

Please could you let us know if we could get an resolution in the coming release:
https://github.com/microsoft/WinAppDriver/issues/1052

[liljohnak]

Thought of another weird thing: .FindElementsByClassName() will return the parent element if the class name maches the search's class name. I do not think that this is the intent that anyone wants so would be nice to remove this behavior if possible. We added a check to to exclude the .Id from the list if it matches the parent.Id

[liljohnak]

We have WinAppDriver failing when the UI thread is locked in the App. We have our own wait that works around this, but maybe consider adding a capability of how long an acceptable ui lock should be.
Bottom of this comment:
https://github.com/microsoft/WinAppDriver/issues/550#issuecomment-456976435

[leticiafer]

@liljohnak Is it possible to use WindowsDriver session = new WindowsDriver(new Uri("http://127.0.0.1:4723"), appCapabilities, TimeSpan.FromMinutes(10)); in java?

[Wolfe1]

@kat-y Any kind of roadmap or developments you can share?

[Purus]

Please remove the Developer Mode requirement. It's a real deal breaker for many organizations.

[DHowett]

Please remove the Developer Mode requirement. It's a real deal breaker for many organizations.

WinAppDriver is a tool that can be trivially used as a remote backdoor to script other applications. The absolute _least_ we can do is require explicit system-level opt-in from an administrator to run it. Anything less, and we are shipping software _signed and trusted by Microsoft_ that malware could drop on a machine undetected and undisturbed. Sorry, but no.

[riverar]

@DHowett That's a very unusual reply, given Microsoft OpenSSH and Visual Studio Remote Debugging tools, both with similar capabilities, don't have the same requirement. Instead of hastily dismissing the request, you should consider that Developer Mode has unintended side effects, like relaxing the need for elevated tokens to create symlinks. Tying WinAppDriver to Developer Mode doesn't quite make sense; I think the request is very appropriate.

[daniel-bsci]

@riverar I agree completely; use other mechanisms to ensure security other than forcing users to put the entire system under test in a less secure state (developer mode). It's not like it's a backdoor to the system; you still need administrator privileges to listen to anything other than local requests.

[specificJyurkiw]

WinAppDriver is a tool that can be trivially used as a remote backdoor to script other applications. The absolute _least_ we can do is require explicit system-level opt-in from an administrator to run it. Anything less, and we are shipping software _signed and trusted by Microsoft_ that malware could drop on a machine undetected and undisturbed. Sorry, but no.

How about some kind of windows container that sandboxes the driver? Would be cool if it could also be used to offer a kind of "effectively headless"-mode since it would basically be a VM anyway. Could also allow us to run tests in parallel on a single computer by running multiple containers so long as they all have their own virtual mice and keyboards.

[specificJyurkiw]

@kat-y Got any updates on WAD development? As far as I can tell it is the ONLY way to automate native windows application testing as the only alternative offered by Microsoft (Coded UI tests) were pulled from Visual Studio in 2019 in favor of _this_ solution, which was then also basically abandoned.

Automation IS how companies and private individuals are testing applications now. And Microsoft is about 7 years behind and getting farther.

I know you can only do so much. Where else should we be making noise to get the right eyes on this problem?

[jlipps]

Hi @kat-y thanks for this update. Glad to hear things are picking up again! Speaking for the Appium maintainers, would it be feasible to have someone from your team represent WAD in our Appium contributor chat group? That way we can communicate easily and make sure changes on both sides maintain compatibility.

[pradeipp]

Hi @kat-y @DHowett, any updates on when we might get a new release? anywhere in 2021 Q2, Q3?

[liljohnak]

@kat-y this comment seems interesting. Maybe merits a doc update? https://github.com/microsoft/WinAppDriver/issues/1464#issuecomment-792452356

[444ghost]

I gave up on waiting for WinAppDriver. It is essentially just a wrapper for MS UI automation. I started building things using that and it works just fine and gives you more degrees of freedom although might take a little bit longer to get familiar with the APIs. This way is better in the long run anyway I think.

[kat-y]

Thank you all so much for your feedback. I don't have a high confidence roadmap right now. I don't mind you asking, so please feel free to @ me now and then, especially on high-priority issues. As I shared previously, the WinAppDriver team has been focused on work to support the UI frameworks and products mentioned above, specifically WinUI 3 as it gears up for its first stable release that will be part of Project Reunion. While supporting WinUI 3 is the highest priority right now, @DHowett and I will be sure to share all updates on this here with you. Thank you so much for being welcoming to me and a patient and understanding community!

[ashkaps]

Hey @kat-y @DHowett as indicated earlier by a few users we have a large community willing to contribute to WAD development.
Most of us are now at a tipping point where we are losing faith because of the sheer amount of delay and no clear vision.
Can MS make this project open source for the community to manage and improve?

[not-the-car]

This means that, for the next ~6+ months, WinAppDriver development will remain paused. In the interim, we’ll be working in the repo with the community to construct an execution plan for when we do bring dev resources back to WinAppDriver.

Hi @kat-y @DHowett

It's been 5 months. Has any work been done reaching out to the community to construct an execution plan?

[specificJyurkiw]

@not-the-car

It's been 5 months. Has any work been done reaching out to the community to construct an execution plan?

Not that I've been able to see.

I already moved on and got Microsoft UI Automation working with my company's WPF application. It's much faster, with fewer dependencies, and it supports things that WAD simply fails to interact with like combo-boxes.

[mattboroi]

@not-the-car

It's been 5 months. Has any work been done reaching out to the community to construct an execution plan?

Not that I've been able to see.

I already moved on and got Microsoft UI Automation working with my company's WPF application. It's much faster, with fewer dependencies, and it supports things that WAD simply fails to interact with like combo-boxes.

Ha, had just requested the combo box issue #389 to be re-opened too.

[Tree55Topz]

@kat-y not to add to the noise, I understand that the mentioned frameworks currently have the focus of your team but what do you mean when you say that you do not have a high confidence roadmap? Does that imply not having high confidence that developers will return to WinAppDriver anytime soon or that the project could be abandoned entirely.

My team of SDETs currently use WinAppDriver for a suite of 400+ UI tests. So far, it has proven to be very stable and flexible but if the project may be abandoned we will need to start looking for alternatives. I really hope that is not the case because the list of tools out there that support coded tests for Windows Applications is not very extensive. I think this could easily dominate that market, especially considering the best one out there currently is LeanFT which is an expensive tool.

[specificJyurkiw]

@Tree55Topz

If you wrote your tests with some kind of interface layer (where that layer interacts with WinAppDriver rather than having the tests interface with it directly), you should be able to trade out WAD for FlaUI and UIA3 in just a few days. I made the swap for 600+ tests in about 4 days and I've been extremely impressed with the performance gains (a test suite that took over an hour to run with WAD now take about 10 minutes to run with FlaUI and UIA3).

[ashkaps]

@specificJyurkiw thanks for making that comment.
Can you tell if FlaUI overcomes the typical challenges of WAD?

1. Slowness in elements identification and execution (already mentioned)
2. Inability to identify certain elements types especially tables and grids
3. Identifying unique element properties in some cases and exposing them like attribute values etc
4. Needing to enable developer mode

[specificJyurkiw]

1. Yes. The performance gains are significant because you don't have to deal with the round trip communication through the WAD server.
2. Not sure about tables and grids, but it did immediately solve the problems I was having with Combo boxes. WAD simply couldn't interface with them effectively except by focusing the element and pressing keys. FlaUI is able to actually grab the child elements.
3. It's about the same as WAD, to be honest. Faster, but the same level of access. AutomationID is, and always will be, your friend.
4. No developer mode dependency.

[liljohnak]

+1 to @specificJyurkiw comment, we also swapped thousands of tests by keeping our test code the same. We only changed the interface libraries code called in the test. Furthermore, we have a few tests running CUIT, White, and WinAppDriver at the same time.
Just don't make your test code dependent on the framework being used, and whatever happens with this project it will be an easy swap to the next framework.

[anunay1]

@specificJyurkiw do you have any code sample or POC for FLAUI with UI3, it will help others if you can share

[ashkaps]

I am ready to help build a java wrapper for FLAUIA3 if anyone is willing.
Something that supports windows frameworks and java too.

[anunay1]

I can write for C#

[ptrthomas]

@ashkaps there's already an open-source effort to wrap UIA3 in Java, do try it out: https://github.com/intuit/karate/wiki/Karate-Robot-Windows-Install-Guide - or feel free to look at the code if you want to use it as a reference

[Tree55Topz]

@specificJyurkiw does FlaUI support XPath selection as well as others like Class, Name, etc? What about location based approaches? Would love to look into it, but the application our tests are built for is rather large and complex. There are not a whole lot of automationIDs to go off of so we rely heavily on Class Name and Name

[specificJyurkiw]

@Tree55Topz Yup. Go take a look at the documentation. The underlying framework is UIA3 which I believe is the same (or is at least related. I can't remember if WAD uses UIA2 or UIA3) underlying framework that WAD uses.

[HenningL]

We developed our own selenium driver on top of snoop for wpf. Was taking 2 man months to have a good solution. Perhaps we open it in the future. Seems like MS doesnt really find a proper solution in short terms.

[liljohnak]

@HenningL We're excited by snoop. Snoop allows for wpf .xaml changes. Also, elements do not need to be added to the AutomationPeer to be seen. Elements only seen by inspect.exe is a drag. I would be interested in this solution.

[jpeltonen]

How about FlaUI with gRPC server? Didn't find any yet. If FlaUI would run with gRPC server then it could be used from any language as gRPC client codes can be generated.

[fforjan]

@kat-y / @DHowett any update on the 6 months pause ? Visual Studio 2022 has been announced and we understand it will NOT support CodedUI. We usually take ~1 year before jumping into a new VS version. It means here we need to prepare our CodedUI migration and we cannot justify usage of Appium to our top level management if this project is paused.

Could a better status/announcement be done (again as you have announced Visual Studio 2022 plan, it should come with) ?

[HenningL]

@HenningL We're excited by snoop. Snoop allows for wpf .xaml changes. Also, elements do not need to be added to the AutomationPeer to be seen. Elements only seen by inspect.exe is a drag. I would be interested in this solution.

@liljohnak As i said. it is not really open but part of our companies services. I just wanted to give a hint, what else can be done.

[PandaMagnus]

1. Yes. The performance gains are significant because you don't have to deal with the round trip communication through the WAD server.
2. Not sure about tables and grids, but it did immediately solve the problems I was having with Combo boxes. WAD simply couldn't interface with them effectively except by focusing the element and pressing keys. FlaUI is able to actually grab the child elements.
3. It's about the same as WAD, to be honest. Faster, but the same level of access. AutomationID is, and always will be, your friend.
4. No developer mode dependency.

I second this. A couple years ago I did a comparison between FlaUI and WAD for a client. We ended up going with WAD because we thought it would be better supported. Features were otherwise comparable, with FlaUI generally being faster unless you really finessed the selectors in WAD (which, frankly, isn't a _bad_ thing, but definitely a learning curve for new developers; but with some finessing we saw comparable performance). Both seem to rely on the UIA library, hence point 3 with same level of access.

I imagine point 2 works because FlaUI seems to provide a more direct abstraction of the UIA library? My motto when testing both was "when in doubt, query UIA." This would let me see the "unfiltered" data before either FlaUI or WAD did it's magic on it, and I seem to recall FlaUI being a little less safe on what it handed back (if you've never messed around with the UIA library, I highly recommend it. You'll learn _a lot_.)

Another alternative is a tool (full disclosure) my buddy is working on to "unit test" XAML. It only works with XAML, so older frameworks are out of luck: https://github.com/Keboo/XAMLTest

[mrlacey]

The OP included the following:

In the interim, we’ll be working in the repo with the community to construct an execution plan for when we do bring dev resources back to WinAppDriver.

What work has been done with the community in the more than 6 months since this issue was opened?

[stu4355226]

I work in a quite some famous company. We were looking for new tools after hearing CodeUI is going to be abandoned... and we found this and converted quite some test cases into WinAppDriver in our CICD pipeline.
Now this tool seems like a dead meat too, and I have to find new tools again, and certainly not MS tools this time.