Pywinauto: Does not work with Win10 Calculator

[sglebs]

Under Windows 10, sending keys to the calculator does not work:

>>> import pywinauto
>>> app = pywinauto.Application().start("calc.exe")
>>> app.Window
<pywinauto.application.WindowSpecification object at 0x00000000024B2860>
>>> app.process
5900
>>> app.Window.TypeKeys("2*3=")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\application.py", line 252, in __getattr__
    ctrls = _resolve_control(self.criteria)
  File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\application.py", line 758, in _resolve_control
    raise e.original_exception
pywinauto.findbestmatch.MatchError: Could not find 'Window' in '[]'
>>> app.Calculator.TypeKeys("2*3=")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\application.py", line 252, in __getattr__
    ctrls = _resolve_control(self.criteria)
  File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\application.py", line 758, in _resolve_control
    raise e.original_exception
pywinauto.findbestmatch.MatchError: Could not find 'Calculator' in '[]'

If I use Task Manager, the Calculator pid is 1940 and not 5900.There is not process with pid 5900.

I think this is causing it to find no windows that belong to this app (it filters by process id first).

On the other hand, Notepad works:

>>> app = pywinauto.Application().start("notepad.exe")
>>> app.process
6600
>>> app.Notepad.TypeKeys("2*3=")
<pywinauto.controls.win32_controls.DialogWrapper object at 0x0000000002B22668>

It does type into notepad.

[sglebs]

Slamming the correct PID does not fix it either:

>>> app = pywinauto.Application().start("calc.exe")
>>> app.process
1208
>>> app.process = 6592
>>> app.process
6592
>>> app.Calculator.TypeKeys("2*3=")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\application.py", line 252, in __getattr__
    ctrls = _resolve_control(self.criteria)
  File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\application.py", line 758, in _resolve_control
    raise e.original_exception
pywinauto.findbestmatch.MatchError: Could not find 'Calculator' in '[]'

[vasily-v-ryabov]

pywinauto doesn't catch spawing additional processes (but we have plans to make windows manipulation available without process binding in the future). In your case you may try app.connect(path='calc.exe') or app.connect(title='Calculator') before finding the top level window.

I've not played with Win10 calculator yet, probably it has no native windows at all. For this case we have UIA branch that is under development (but feedback is welcome). After installing comtypes package you will be able to run app = pywinauto.Application(backend='uia').start('calc.exe') that will use Microsoft UI Automation technology instead of native Win32 API (0.5.4 supports Win32 API only). Other high-level API should remain almost the same.

[sglebs]

Thanks! Connecting to calc.exe fails:

>>> app.connect(path='calc.exe')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\application.py", line 829, in connect
    self.process = process_from_module(kwargs['path'])
  File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\application.py", line 1266, in process_from_module
    raise ProcessNotFoundError(message)
pywinauto.application.ProcessNotFoundError: Could not find any process with a module of 'calc.exe'

Connecting to calc seems to work at connecting, but sending keys does not work:

>>> app.connect(path='calc')
<pywinauto.application.Application object at 0x0000000002337550>
>>> app.Window.TypeKeys("2*3=")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\application.py", line 252, in __getattr__
    ctrls = _resolve_control(self.criteria)
  File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\application.py", line 758, in _resolve_control
    raise e.original_exception
pywinauto.findbestmatch.MatchError: Could not find 'Window' in '[]'

By title works! But I have to use app.Calculator and not app.Window:

>>> app.Calculator.TypeKeys("2*3=")
<pywinauto.controls.win32_controls.DialogWrapper object at 0x0000000002584080>

I will take a look at the UIA stuff, thanks!

Question: I want to have a utility function to do the work, so app.Calculator does not fly for me. How can I do the same but passing "Calculator" as parameter? Do I have to call findbestmatch directly or is there a nicer way? Thanks!

[sglebs]

Never mind, found it in the source code:

Once you have an Application instance you can access dialogs in that
application either by using one of the methods below. ::

   dlg = app.YourDialogTitle
   dlg = app.ChildWindow(title = "your title", classname = "your class", ...)
   dlg = app['Your Dialog Title']

[sglebs]

I tried the UIA as you suggested:

(env) C:\Users\marcio\git\webdriver-python>pip install --upgrade git+https://github.com/pywinauto/pywinauto.git@UIA
Collecting git+https://github.com/pywinauto/pywinauto.git@UIA
  Cloning https://github.com/pywinauto/pywinauto.git (to UIA) to c:\users\marcio\appdata\local\temp\pip-iuihqp-build
Installing collected packages: pywinauto
  Found existing installation: pywinauto 0.5.4
    Uninstalling pywinauto-0.5.4:
      Successfully uninstalled pywinauto-0.5.4
  Running setup.py install for pywinauto ... done
Successfully installed pywinauto-0.5.4

(env) C:\Users\marcio\git\webdriver-python>python
Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:24:40) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pywinauto
2016-07-07 09:50:54,828 INFO: Could not import comtypes.gen, trying to create it.
2016-07-07 09:50:54,828 INFO: Created comtypes.gen directory: 'C:\Users\marcio\git\webdriver-python\env\lib\site-packages\comtypes\gen'
2016-07-07 09:50:54,828 INFO: Writing __init__.py file: 'C:\Users\marcio\git\webdriver-python\env\lib\site-packages\comtypes\gen\__init__.py'
2016-07-07 09:50:54,859 INFO: Using writeable comtypes cache directory: 'C:\Users\marcio\git\webdriver-python\env\lib\site-packages\comtypes\gen'
2016-07-07 09:50:54,875 INFO: Could not import comtypes.gen._944DE083_8FB8_45CF_BCB7_C477ACB2F897_0_1_0: No module named _944DE083_8FB8_45CF_BCB7_C477ACB2F897_0_1_0
# Generating comtypes.gen._944DE083_8FB8_45CF_BCB7_C477ACB2F897_0_1_0
2016-07-07 09:50:54,969 INFO: Could not import comtypes.gen._00020430_0000_0000_C000_000000000046_0_2_0: No module named _00020430_0000_0000_C000_000000000046_0_2_0
# Generating comtypes.gen._00020430_0000_0000_C000_000000000046_0_2_0
2016-07-07 09:50:55,046 INFO: Could not import comtypes.gen.stdole: No module named stdole
# Generating comtypes.gen.stdole
2016-07-07 09:50:55,234 INFO: Could not import comtypes.gen.UIAutomationClient: No module named UIAutomationClient
# Generating comtypes.gen.UIAutomationClient
>>>

Then I tried this, and it successfully launched it:

app = pywinauto.Application(backend='uia').start('calc.exe') 

But this took forever and then failed:

>>> app.top_window_()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\application.py", line 1039, in top_window_
    raise RuntimeError("No windows for that process could be found")
RuntimeError: No windows for that process could be found

This also fails:

>>> app.Calculator.TypeKeys("2*3=")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\application.py", line 344, in __getattribute__
    ctrls = self.__resolve_control(self.criteria)
  File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\application.py", line 221, in __resolve_control
    raise e.original_exception
pywinauto.findwindows.ElementNotFoundError: {'process': 2224, 'best_match': 'Calculator', 'backend': u'kia'}

Very strange the backend='kia' if I passed backend='uia' when starting. (I was not able to make it happen again (printed via later). It must have been a console copy/paste problem.)

[vasily-v-ryabov]

Thanks for the detailed feedback.
Is connect by title working for backend='uia'? If yes, you may try app.Calculator.print_control_identifiers(depth=5). It should print access names for the second level of attribute access. Now it works as described here, but I'm implementing 3+ level window specifications right now.

[sglebs]

I thought that UIA would allow me to eliminate the need to connect. If I need to connect regardless, then I am not sure why I would want UIA. Is it better? For owner-drawn UIs I guess?

Connecting "worked"(no exception) but did not send the keystrokes:

>>> app = pywinauto.Application(backend='uia').start('calc.exe')
>>> app.connect(title='Calculator')
<pywinauto.application.Application object at 0x0000000003CD14E0>
>>> app.Calculator.TypeKeys("2*3=")
<pywinauto.controls.uiawrapper.UIAWrapper object at 0x0000000003CD1978>

So, the only scenario that has worked for me (doing some math with the calculator) is with the oficial release and using connect:

>>> import pywinauto
>>> app = pywinauto.Application().start('calc.exe')
>>> app.connect(title='Calculator')
<pywinauto.application.Application object at 0x0000000002927550>
>>> app.Calculator.TypeKeys("2*3=")
<pywinauto.controls.win32_controls.DialogWrapper object at 0x00000000031ED7B8>

BTW, this same scenario with the UIA branch (but not using the via backend) also fails, with an exception:

>>> import pywinauto
2016-07-07 11:27:03,743 INFO: Imported existing <module 'comtypes.gen' from 'C:\Users\marcio\git\webdriver-python\env\lib\site-packages\comtypes\gen\__init__.pyc'>
2016-07-07 11:27:03,759 INFO: Using writeable comtypes cache directory: 'C:\Users\marcio\git\webdriver-python\env\lib\site-packages\comtypes\gen'
>>> app = pywinauto.Application().start('calc.exe')
>>> app.connect(title='Calculator')
<pywinauto.application.Application object at 0x0000000002A87550>
>>> app.Calculator.TypeKeys("2*3=")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\base_wrapper.py", line 796, in type_keys
    self.set_focus()
  File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\controls\hwndwrapper.py", line 1183, in set_focus
    1)
pywintypes.error: (5, 'AttachThreadInput', 'Access is denied.')

[vasily-v-ryabov]

UIA is better for WPF, WinForms and Store apps (you can do complicated things without type_keys workarounds). Process-based window manipulation is used in both branches for now. Of course, UIA is not better for native apps now because it's not release ready. After some fixes it should be better I hope.

pywintypes.error: (5, 'AttachThreadInput', 'Access is denied.')

Interesting. Did you try running as Administrator?

[sglebs]

Just found an article which details the Calc issue: https://stackoverflow.com/questions/37130985/is-autoit-able-to-interact-with-ui-controls-in-windows-store-apps

[sglebs]

just for the record, the pyuiautomation example did not work for me https://github.com/zerg-rush/pyuiautomation/issues/3

[sglebs]

Regarding your question abou the "Access is denied", I opened the shell again as Administrator, entered the virtualenv again, tried it again using the UIA branch but not with backend-'via', and got a different error:

>>> app.Calculator.TypeKeys("2*3=")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\base_wrapper.py", line 796, in type_keys
    self.set_focus()
  File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\controls\hwndwrapper.py", line 1183, in set_focus
    1)
pywintypes.error: (87, 'AttachThreadInput', 'The parameter is incorrect.')

[airelil]

I also got the same error in set_focus recently. Tried to make a small
automation to press OK button in a VS2010 dialog but set_focus always
failed. (UIA branch, Py34 x86)

On Fri, 8 Jul 2016, 05:39 Marcio Marchini, [email protected] wrote:

Regarding your question abou the "Access is denied", I opened the shell
again as Administrator, entered the virtualenv again, tried it again using
the UIA branch but not with backend-'via', and got a different error:

app.Calculator.TypeKeys("2*3=")
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\base_wrapper.py", line 796, in type_keys
self.set_focus()
File "C:\Users\marcio\git\webdriver-python\env\lib\site-packages\pywinauto\controls\hwndwrapper.py", line 1183, in set_focus
1)
pywintypes.error: (87, 'AttachThreadInput', 'The parameter is incorrect.')

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/pywinauto/pywinauto/issues/209#issuecomment-231152244,
or mute the thread
https://github.com/notifications/unsubscribe/AAwhoHcY2snhQU6EoUaSJvO2QZh7nVkGks5qTTm3gaJpZM4JFYk4
.

Cheers,
Valentin

[vasily-v-ryabov]

I've added an ability to call something like Desktop(backend='uia').Calculator.type_keys('2*3=') in #211. But the problem with AttachThreadInput will be fixed later. As an alternative you can use the following workaround in UIA branch:

from pywinauto import keyboard
keyboard.type(u'2*3=')

It doesn't set any focus.

[vasily-v-ryabov]

After a bit more detailed investigation I've figured out the root cause is a minimized state of the window. If the window is minimized set_focus() fails even for Notepad and likely for any app. We need to implement checking minimized/restored/maximized state of the window and calling particular method to restore/maximize the window.

[vasily-v-ryabov]

The problem is not only in minimized state, but I can reproduce it in UIA very easy:

app = Application(backend="win32").connect(title="Calculator")
app.Calculator.minimize()
app.Calculator.type_keys('2*3=')

while without app.Calculator.minimize() it works.

[vasily-v-ryabov]

Hmm... Interesting. Calculator.exe process status is Suspended while the window is minimized. For usual apps all the processes are in Running state. That might be a reason why first minimization blocks Win32 API input thread.

[airelil]

@vasily-v-ryabov , are you talking about UIA? Because your example above specifies the win32 backend. Edit the comment if it's UIA. Did you try to restore the window if it's minimized from inside set_focus ?

[vasily-v-ryabov]

It's for UIA branch, but for win32 backend. The code snippet is correct.
And yes, I tried to restore the window inside set_focus (it didn't help).

[vasily-v-ryabov]

For backend="uia" we have another problem: minimized Calculator has only 1 automation element with name "Calculator" but it's off-screen. Inspect.exe can see this element, but pywinauto doesn't list it as an available window. So in this case we cannot restore the window.

[vasily-v-ryabov]

OK, after several improvements since 0.6.0 the following example works on Win10:

from pywinauto import Desktop, Application

app = Application(backend="uia").start('calc.exe')

dlg = Desktop(backend="uia").Calculator
dlg.type_keys('2*3=')
dlg.print_control_identifiers()
dlg.minimize()

Desktop(backend="uia").window(title='Calculator', visible_only=False).restore()

I created example win10_calculator.py with this code. It works good enough so that we can close this issue. Any objections?

[vasily-v-ryabov]

OK, going to close this thread. @sglebs, if you believe some improvements are still necessary please re-open it or file a new issue. Thanks!

P.S. pywinauto 0.6.1 is out!

[karanphol]

thank you @vasily-v-ryabov is good guideline