i have a very simple script that click few buttons in calculator app. If I use backend win32 executing script takes about 16 sec, but if I use backend uia same script takes 1 min 35 sec. Below examples scripts. Each script is executed 4 times.
script below uses buttons text to find it
app = Application()
appconnect = app.connect(title='Calculator')
app_dlg = appconnect.top_window_()
app_dlg.window_(best_match='3').Click()
script below uses buttons automatinID
app = Application(backend='uia')
appconnect = app.connect(title='Calculator')
app_dlg = appconnect.top_window_()
app_dlg.child_window(auto_id='button5').click_input()
Choosing search criteria is important for performance. I'd recommend using class_name='Button' in child_window because getting class_name works faster and it filters out a lot of unnecessary controls before searching by auto_id or by title. Though this is true for backend='win32' as well.
it did not work in this case. Is there any other way to get this go bit faster? now it still takes 13-14 sec and with backend='win32' takes 1 sec.
app = Application(backend='uia')
appconnect = app.connect(title='Calculator')
app_dlg = appconnect.top_window_()
app_dlg.child_window(class_name='WindowsForms10.BUTTON.app.0.141b42a_r14_ad1', auto_id='button5').click_input()
Hmm... I think I can improve this for auto_id case (will commit soon). Thanks for the report.
actually it seems that connect() and top_window_() methods are taking the most time to complete. connect() method takes 6-7 sec and top_window_() takes same 6-7 sec
appconnect = app.connect(title='Calculator')
app_dlg = appconnect.top_window_()
They both work for 0.042 sec for me. How many windows do you have opened on your desktop? Is that time reduced after closing some of them?
did not help. only have python and test app. also noticed something interesting
code below
app = Application(backend='uia')
appconnect = app.connect(title='Calculator', framework_id='WinForm')
app_dlg = appconnect.top_window_()
app_dlg.child_window(class_name='WindowsForms10.BUTTON.app.0.141b42a_r14_ad1', auto_id='button5', title='3').click_input()
times to execute
2016-10-29 23:20:49.516000 start
2016-10-29 23:20:49.579000 backend selected
2016-10-29 23:20:55.710000 connection made
2016-10-29 23:21:01.809000 top window
2016-10-29 23:21:02.184000 button clicked and end
code below, I changed top_window_() method to window_() method and now it takes more time to find the actual button
app = Application(backend='uia')
appconnect = app.connect(title='Calculator', framework_id='WinForm')
app_dlg = appconnect.Window_(title='Calculator')
app_dlg.child_window(class_name='WindowsForms10.BUTTON.app.0.141b42a_r14_ad1', auto_id='button5', title='3').click_input()
time to execute
2016-10-29 23:21:59.436000 start
2016-10-29 23:21:59.483000 backend selected
2016-10-29 23:22:05.723000 connection made
2016-10-29 23:22:05.738000 window
2016-10-29 23:22:12.165000 button clicked and end
@nimuston, can you switch to path argument in the connect call ? Like this:
app.connect(path="calculator.exe")
wow now it only takes 7 sec :) this path is now searching calculator.exe process name and makes connection base on that? is this correct? is there still some ways to make it a bit more faster?
2016-10-29 23:40:07.608000 start
2016-10-29 23:40:07.624000 backend selected
2016-10-29 23:40:07.796000 connection made
2016-10-29 23:40:14.644000 top window
2016-10-29 23:40:14.956000 button clicked and end
path is now searching calculator.exe process name and makes connection base on that? is this correct?
Yes, you can specify an executable by its name
is there still some ways to make it a bit more faster
You should try to specify more detailed search criteria. Maybe you could use print_control_identifiers to find more clues:
app.top_window().print_control_identifiers()
and after that try to build a call like:
app.window(title="Calculator", class_name="...")
still some problems with slowness even if I use multiple search criteria's for the button. Even if I use buttons title or window text. I see that this is labeled as bug so any idea when the fix is available?
Hmm... I think I can improve this for auto_id case (will commit soon). Thanks for the report.
example with the test app WpfApplication1.exe it takes 5 sec to press one button. Same operation with backend='win32' takes less then a seconds.
app = Application(backend='uia')
appconnect = app.connect(path='WpfApplication1.exe')
dlg = appconnect.window_(title='WPF Sample Application')
dlg.child_window(auto_id='button2', title='Apply').type_keys('ENTER')
@nimuston, UIA backend always runs slower than the native 'win32', but for this simple example it shouldn't take 5 seconds. On my machine it finishes in 200 mlsec.
Could you run the code below and show us the output? I'm just curious to see the timing.
import logging
from pywinauto.application import Application
# Config logger
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
def run_app(backend):
logger.info("Start UIA")
app = Application(backend=backend)
logger.info("Created")
appconnect = app.connect(path='WpfApplication1.exe')
logger.info("Connected")
dlg = appconnect.window_(title='WPF Sample Application')
logger.info("Dialog win spec is created")
btn_wrp = dlg.child_window(auto_id='button2', title='Apply').wrapper_object()
logger.info("Button wrapper is created")
btn_wrp.type_keys('ENTER')
logger.info("End UIA")
run_app('uia')
results
2016-11-01 14:43:21,242 INFO: Start UIA
2016-11-01 14:43:21,242 INFO: Created
2016-11-01 14:43:21,256 DEBUG: Release <POINTER(ITypeLib) ptr=0xccbc20 at 3b69148>
2016-11-01 14:43:21,319 INFO: Connected
2016-11-01 14:43:21,319 INFO: Dialog win spec is created
2016-11-01 14:43:21,319 DEBUG: Release <POINTER(IUIAutomationCondition) ptr=0x4243940 at 40cc848>
2016-11-01 14:43:21,319 DEBUG: Release <POINTER(IUIAutomationCondition) ptr=0x30eeee0 at 4012048>
2016-11-01 14:43:27,039 DEBUG: Release <POINTER(IUIAutomationElementArray) ptr=0x4241d70 at 4012048>
2016-11-01 14:43:27,039 DEBUG: Release <POINTER(IUIAutomationCondition) ptr=0x42439f0 at 3fe3048>
2016-11-01 14:43:27,164 DEBUG: Release <POINTER(IUIAutomationElementArray) ptr=0x4241d10 at 4012048>
2016-11-01 14:43:27,164 DEBUG: Release <POINTER(IUIAutomationCondition) ptr=0x4245070 at 3fe3048>
2016-11-01 14:43:27,164 INFO: Button wrapper is created
2016-11-01 14:43:27,226 INFO: End UIA
2016-11-01 14:43:27,226 DEBUG: Calling CoUnititialize()
2016-11-01 14:43:27,226 DEBUG: CoUnititialize() done.
OK, now it's clear that btn_wrp = dlg.child_window(auto_id='button2', title='Apply').wrapper_object() takes long.
Try this criteria:
btn_wrp = dlg.child_window(auto_id='button2', title='Apply', control_type='Button').wrapper_object()
What is the output then?
it did not get any faster, but this would be ok for me that the pressing button takes 4-7 sec, but the real problem I have is with the app that I'm actually automating. It takes 4-8 minutes to press buttons. This is really bad thing. The app that I'm automating contains multiply winforms and multiple buttons and it seems that if I'm example in the main view and I press button, then app opens another winforms inside the main form and if I try to press another button there it takes really long time to execute. Any idea how I can get it to be faster? it seems that now it goes through every time every buttons until it finds a match. Is there a way that I can tell to the script that this is the main forms example "app.title" and that is should only search button inside "second.winform" and maybe it would be faster that way?
I already tried something like this, did not work for me
window = app.window(title='app.title')
secondwindow = window.child_window(title="second.winform", control_type="Window")
secondwindow.child_window(auto_id='button1').type_keys('{ENTER}')
2016-11-01 19:42:34,819 INFO: Start UIA
2016-11-01 19:42:34,819 INFO: Created
2016-11-01 19:42:35,051 INFO: Connected
2016-11-01 19:42:35,052 INFO: Dialog win spec is created
2016-11-01 19:42:35,052 DEBUG: Release <POINTER(IUIAutomationCondition) ptr=0x2e
f910 at 2d060c8>
2016-11-01 19:42:35,053 DEBUG: Release <POINTER(IUIAutomationCondition) ptr=0x2e
f860 at 2d0c848>
2016-11-01 19:42:39,572 DEBUG: Release <POINTER(IUIAutomationElementArray) ptr=0
x2eb670 at 2d0c848>
2016-11-01 19:42:39,572 DEBUG: Release <POINTER(IUIAutomationCondition) ptr=0x2e
f9c0 at 2dd8848>
2016-11-01 19:42:39,575 DEBUG: Release <POINTER(IUIAutomationCondition) ptr=0x46
8d50 at 2dd8848>
2016-11-01 19:42:39,575 DEBUG: Release <POINTER(IUIAutomationCondition) ptr=0x46
8ca0 at 2d0c848>
2016-11-01 19:42:39,767 DEBUG: Release <POINTER(IUIAutomationElementArray) ptr=0
x2eb670 at 2d0c848>
2016-11-01 19:42:39,769 DEBUG: Release <POINTER(IUIAutomationCondition) ptr=0x46
8e00 at 2dd8f48>
2016-11-01 19:42:39,770 INFO: Button wrapper is created
2016-11-01 19:42:39,792 INFO: End UIA
2016-11-01 19:42:39,793 DEBUG: Calling CoUnititialize()
2016-11-01 19:42:39,793 DEBUG: CoUnititialize() done.
For the child window form it's possible to use parent keyword in search criteria.
Say you have winform2 = app.winform1.winform2.wait('ready') # returns UiaWrapper.
Then you may create criteria so: app.winform1.child_window(parent=winform2, control_type='Button', title='Apply'). Yeah, it's a little bit bulky but should work. Will think about getting window specifications from wrappers as well. It might be a good idea.
tried that and got TypeError: descendants() got an unexpected keyword argument 'process' maybe I just don't understand this the right way.
This looks like a serious bug. Thanks! We will fix it hopefully this week.
hi, any progress with the issue? any possibility to make uia work faster? it is very slow if I have multiply winforms in the same app opened same time. :(
@nimuston, if your winforms are constantly open you can keep wrappers for each form or even for buttons on the forms and avoid additional windows resolution.
# Find forms and keep
frm1_wrp = app.WinForm1.wrapper_object()
frm2_wrp = app.WinForm2.wrapper_object()
# locate the buttons on the forms
btn_lst = fm1_wrp.descendants(control_type="Button")
btn_lst[0].click_input()
btn_lst = fm2_wrp.descendants(control_type="Button")
btn_lst[0].click_input
update: grammar
@nimuston, I understand your impatience, but our main jobs aren't related to GUI automation. So bug fixing might be slow. Please keep it in mind.
I've fixed the issue with parent criterion locally, but I need to write more tests before merging to the main repo. It will take some time.
So far you can try this workaround:
app.winform1.child_window(parent=winform2.element_info, control_type='Button', title='Apply')
I'll make calling element_info property unnecessary in the next pull request.
P.S. Your feedback is really very useful! We appreciate it. 馃憤
Hi, good that my feedback is useful :) I tried something like this, correct my if my code is totally wrong, also i got a nice error message. It also seems that with the app i'm trying to automate it takes really long time to print control indentifiers, it might be because the app has "multiple" layers. I meant that it has a lot of winforms on top of each other.
from pywinauto.application import Application
app = Application(backend='uia')
appconnect = app.connect(title='app.title')
window = appconnect.window(title='app.title')
window2= window.child_window(title="3", auto_id="winformname", control_type="Window")
button = window2.child_window(parent=window, control_type='button', auto_id='_buttonname_1')
bwp = button.wrapper_object()
Traceback (most recent call last):
File "<pyshell#9>", line 1, in <module>
bwp = button.wrapper_object()
File "build\bdist.win-amd64\egg\pywinauto\application.py", line 246, in wrapper_object
ctrls = self.__resolve_control(self.criteria)
File "build\bdist.win-amd64\egg\pywinauto\application.py", line 236, in __resolve_control
criteria)
File "build\bdist.win-amd64\egg\pywinauto\timings.py", line 402, in wait_until_passes
func_val = func(*args)
File "build\bdist.win-amd64\egg\pywinauto\application.py", line 199, in __get_ctrl
ctrl = self.backend.generic_wrapper_class(findwindows.find_element(**ctrl_criteria))
File "build\bdist.win-amd64\egg\pywinauto\findwindows.py", line 84, in find_element
elements = find_elements(**kwargs)
File "build\bdist.win-amd64\egg\pywinauto\findwindows.py", line 207, in find_elements
cache_enable=True)
TypeError: descendants() got an unexpected keyword argument 'process'
Please change this line:
button = window2.child_window(parent=window, control_type='button', auto_id='_buttonname_1')
to this one:
button = window2.child_window(parent=window.element_info, control_type='button', auto_id='_buttonname_1')
hi, @vasily-v-ryabov i changed the code but it did not get any faster, also i'm wondering do you know why print_control_identifiers() takes over 2 minutes to complete even if i use depth=1 print_control_identifiers(depth=1) ?
Also can i somehow use this depth definition when i search button?
We already saw in your traces that there is a 5-second gap to locate the main window of the application. Notice the time stamp at the last line
2016-11-01 14:43:21,319 INFO: Connected
2016-11-01 14:43:21,319 INFO: Dialog win spec is created
2016-11-01 14:43:21,319 DEBUG: Release <POINTER(IUIAutomationCondition) ptr=0x4243940 at >40cc848>
2016-11-01 14:43:21,319 DEBUG: Release <POINTER(IUIAutomationCondition) ptr=0x30eeee0 at 4012048>
2016-11-01 14:43:27,039 DEBUG: Release <POINTER(IUIAutomationElementArray) ptr=0x4241d70 at 4012048>
I think this is the root cause. Even print_control_identifiers start with a top window. It's not clear, at least for me, why on your machine it takes so long.
In so called "real use" the gap is bigger 1-2 minutes
Can you provide some details about your environment: OS, architecture, python version e.t.c. ? Did you try to tune it somehow ? Things like: disabling antivirus, raising the python process priority....
Win 10 64 bit, python 2.7 i'm running python script from command line and priority for that window is highest. Tested all the possible search criterias
Hi, found another way to do this it is 1 sec faster :) but still not fast enough
app = pywinauto.Application(backend='uia')
appconnect = app.connect(path='app.exe')
window = app.top_window()
wrapper = window.wrapper_object()
ctrl = wrapper.descendants(control_type='Button')
button = ctrl[1].click_input()
Is there a way for me to know what actually is inside every index of this list? now if i print for example one index from the list is shows me this below:
<pywinauto.controls.uia_controls.ButtonWrapper object at 0x0000000004D26390>
But yeah this is not the right way to do it, just trying to find ways to find the buttons faster
Is there a way for me to know what actually is inside every index of this list?
print [ctrl.window_text() for ctrl in wrapper.descendants(control_type='Button')]
Hi again,
@vasily-v-ryabov i tested the fix you make for "findwindows.py" file and i got this new error message:
TypeError: descendants() got an unexpected keyword argument 'cache_enable'
do you have time to check what is wrong?
Nice i accidentally closed this issue, how i can open it again?
So the original problem " TypeError: descendants() got an unexpected keyword argument 'process' seems to be fixed in this new findwindows.py file " , but it generated new error " TypeError: descendants() got an unexpected keyword argument 'cache_enable' "
@nimuston, can you show the full exception stack. Which commit did you use exactly ?
BTW, as I understand you still have the problem with the slow access to the top window. I just thought about an additional option with using Desktop object. For example, if I have an instance of Process Explorer, I can access it as following:
from pywinauto import Desktop
d = Desktop(backend='uia')
d.ProcessExplorerSysinternalsWwwSysinternalsCom.draw_outline()
code I'm using:
app = pywinauto.Application(backend='uia')
appconnect = app.connect(path='WpfApplication1.exe')
appwindow = appconnect.window(title="WPF Sample Application", control_type="Window")
secondwindow = appwindow.child_window(title="Alpha", control_type="ToolBar")
button = secondwindow.child_window(parent=secondwindow, title="button 1", auto_id="toolbar_button1", control_type="Button")
button.click_input()
error message:
Traceback (most recent call last):
File "<pyshell#11>", line 1, in <module>
button.click_input()
File "build\bdist.win-amd64\egg\pywinauto\application.py", line 346, in __getattribute__
ctrls = self.__resolve_control(self.criteria)
File "build\bdist.win-amd64\egg\pywinauto\application.py", line 236, in __resolve_control
criteria)
File "build\bdist.win-amd64\egg\pywinauto\timings.py", line 402, in wait_until_passes
func_val = func(*args)
File "build\bdist.win-amd64\egg\pywinauto\application.py", line 199, in __get_ctrl
ctrl = self.backend.generic_wrapper_class(findwindows.find_element(**ctrl_criteria))
File "build\bdist.win-amd64\egg\pywinauto\findwindows.py", line 85, in find_element
elements = find_elements(**kwargs)
File "build\bdist.win-amd64\egg\pywinauto\findwindows.py", line 209, in find_elements
cache_enable=True)
TypeError: descendants() got an unexpected keyword argument 'cache_enable'
and yes I still have a problem with the slow access to the top window. I tried using Desktop object but the results were same still slow.
actually it seems that Desktop object is a bit faster than Application object. at least with the test app
def desk():
start = time.time()
app = Desktop(backend='uia')
window = app.window(title='WPF Sample Application')
window.child_window(auto_id='button2').click_input()
print time.time() - start
def app():
start = time.time()
app = Application(backend='uia')
appconnect = app.connect(title='WPF Sample Application')
appwindow = appconnect.window()
appwindow.child_window(auto_id='button2').click_input()
print time.time() - start
in this case desk() takes 7,2 sec to run and app() takes 13,8 sec to run, so I really need to test this in real use, thanks!
I will take the previous comment back, run time is almost the same in both cases, if i just change (title='WPF Sample Application') to (path='WpfApplication1.exe') in app function.
@numiston, here a fixed example. Notice about the parent argument
app = pywinauto.Application(backend='uia')
appconnect = app.connect(path='WpfApplication1.exe')
appwindow = appconnect.window(title="WPF Sample Application", control_type="Window")
secondwindow = appwindow.child_window(title="Alpha", control_type="ToolBar")
button = secondwindow.child_window(parent=secondwindow.wrapper_object().element_info, title="button 1", auto_id="toolbar_button1", control_type="Button")
button.click_input()
@vasily-v-ryabov, there is a problem with passing a WindowSpecification or a wrapper object directly as the parent argument into findwindow chain. We may fix it in find_elements
Hi @nimuston, thanks for the report! And @airelil thanks for the fix!
I hope to get back with some optimizations after these long New Year holidays. :)
@nimuston, I know it's doesn't help with your main problem but still. With my fix in the master you can get now use your original code without specifiying 'element_info' explicitly:
button = secondwindow.child_window(parent=secondwindow, title="button 1", auto_id="toolbar_button1", control_type="Button")
button.click_input()
@airelil your fix works thanks!
yeah the original slowness problem is the biggest issue i have at the moment. I was thinking would it be faster if we were able to use example search depth when we are searching buttons auto_id. Something like this button = secondwindow.child_window(parent=secondwindow, title="button 1", auto_id="toolbar_button1", control_type="Button", searchdepth='3') . One idea to play around :)
Hello,
I see you spend time to improve speed.
But i wonder why these two methodes take the same time to execute:
#time find_till_btn()
#Wall time: 35.1 s
def find_till_btn():
app_uia = application.Application(backend='uia').connect(path='TPDotnet.Pos.exe')
win = app_uia['TP.net POS']
win.child_window(auto_id='_cmdFunctionKey_13').wait('visible', timeout=60)
# with control_type filter
#Wall time: 35.7 s
def find_till_btn2():
app_uia = application.Application(backend='uia').connect(path='TPDotnet.Pos.exe')
win = app_uia['TP.net POS']
win.child_window(auto_id='_cmdFunctionKey_13', control_type='Button').wait('visible', timeout=60)
Hi @sragons thanks for testing it. Do you use 0.6.2 or current master branch? We made some fixes to check control_type prior to auto_id some time ago (but after 0.6.2 was out).
I use the master branch.
Are your fixes present on that branch ?
So my request is relevant ?
I also encountered a performance issue with the uia backend on my win10 box (5 seconds delay for finding main window); in my case the culprit seems to be the COM IUIAutomationElement::FindAll call. Eg in
Desktop("uia").window(class_name="Chrome_WidgetWin_1", title_re=".\*Visual Studio Code.\*").set_focus()
it seems to take FindAll 5 seconds to get to the right window from root, even though TreeScope is set to "children". (see UIAElementInfo.children(..) in uia_element_info.py)
I will try to dig further, but meanwhile, any insight in how/why this could happen?
Hi @RemcoTukker can you try the same on current master?
Direct call to UIAElementInfo().children() also takes the same time?
@vasily-v-ryabov Yes, it is the same on the current master, and I'm sure now that my particular problem is not caused by the pywinauto code. It's the COM FindAll on the root that is slow. I even tried to use it from a small C++ program, and still the same result.
After that I tried to use a TreeWalker, which shows again a similar delay of 4 seconds at the moment I use GetNextSiblingElement on the last child. So, 'FindFirst' functionality would be a possible workaround.
I tried the same on a win7 machine, and there everything worked fine. I don't have access to other win10 machines right now to test if it is just me / some weird stuff in my environment. For reference, I'm using Win10 home edition (up-to-date), but not the creators update.
So what's left to do? Maybe report it to MS somewhere?
I had some fun with writing a monkey patch using the "FirstChild" idea, see this gist. This works around the FindAll call that is slow on my machine.
@nimuston @sragons If you want, you can try if using the gist makes a difference for you. Maybe you're looking at the same problem?
@vasily-v-ryabov Let me know if you think this functionality is good to have for other people as well. If so, I can make a proper pull request for it.
@RemcoTukker I did some testing with your fix and it seems that is it actually faster. For example
d1.window(first_only=True, title="TestApp").set_focus() takes 0,9 sec to execute
d1.window(title="TestApp").set_focus() and previously it took 3 sec to execute
@RemcoTukker this is great idea to add first_only=True option. 馃憤
It's worth to say option found_index=0 exists for all backends but it doesn't use FindFirst/GetNextSibling as well (I think it should be kept separate but it's discussable). It's just another way to disambiguate search criteria inherited from backend='win32' and it was never optimized. Ideally all backends should be in sync from class ElementInfo point of view for consistency. I'd be happy to discuss all these points in a pull request.
Thank you guys for your efforts!
@RemcoTukker Any plans to add other search criteria to the fix so that we can see how fast this fix really is? example auto_id
Yes I will prepare a pull request.. give me a couple of days though, i'm trying to fit it in office hours. As for your results, I think 0.9 seconds is still slow, and part of the gain you're seeing may be explained by not having to check all the windows. So I'm not sure it will solve all your problems @nimuston , but hopefully its a good start.
I was taking another look at the code today, and now I noticed you can already pass in the handle to the window function, which at least on my win10 pc is very fast. It completely avoids the slow COM UIA calls as long as the handle doesnt change. I guess that the handle stays constant for most usecases (in particular on the root level).
So maybe the best thing to do would be to recommend users to cache the handle themselves (or maybe with some help from pywinauto)? My workaround code would then only make the initial lookup a bit faster, but Im not sure that benefit is worth the extra mess in the code...
edit Ah wait, caching the handle is basically the same as caching the wrapper_object, right?
edit Ah wait, caching the handle is basically the same as caching the wrapper_object, right?
Correct. Not every UI element has native handle, but in any case wrapper_object isn't re-created if you use return value of .wrapper_object() twice or more.
Right, all is clear to me now, I'll get to work :)
In conclusion, as you mentioned earlier, the most significant speedup for me is to cache the wrapper_object and then use it like this:
pywinauto.Desktop('uia').window(parent=cached_wrapper_object, title_re='something.*', top_level_only=False)
Adding the process, class_name, control_type and title to the criteria can also help.
Maybe it's a good idea to document the above somewhere or to make it easier to do this, eg by allowing calling .window() on a wrapper object, as you proposed. Apart from that I will make a pull request for a slightly optimized version for finding the first direct child that satisfies some criteria.
@vasily-v-ryabov Yes, it is the same on the current master, and I'm sure now that my particular problem is not caused by the pywinauto code. It's the COM
FindAllon the root that is slow. I even tried to use it from a small C++ program, and still the same result.
I had the same problem with C ++ code. This is Windows 10 issue, and should be resolved by Windows 10 update kb3093266.
Most helpful comment
@nimuston, can you switch to
pathargument in theconnectcall ? Like this: