Pywinauto: target position is wrong for _perform_click in some cases

[nixgnef]

Expected Behavior

pywinauto.controls.hwndwrapper._perform_click should simulate the click behavior correctly.

Actual Behavior

when the “Dispaly -> Scale and layout -> Change the size of text, apps and other items” setting for windows 10 OS is NOT 100%， the target position of
pywinauto.controls.hwndwrapper._perform_click
may need to be adjusted according to the scale ratio.

Steps to Reproduce the Problem

1. set “dispaly -> scale and layout -> change the size of text, apps and other items” to be 200%

2. use pywinauto with win32 backend to get an item from a SysTreeView32 control in an app ，and then call the click() method for this item

Short Example of Code to Demonstrate the Problem

            tree_view = app.top_window().child_window(
                control_id=129, class_name="SysTreeView32"
            )
            target_item = tree_view.wrapper_object().get_item(['query','target'])
            target_item.click()

A raw method to solve this problem

1. add import win32print into hwndwrapper.py

2. insert the following code into pywinauto.controls.hwndwrapper._perform_click near line 1663 in hwndwrapper.py

    scale_ratio = win32print.GetDeviceCaps(win32gui.GetDC(ctrl.handle), win32con.LOGPIXELSX) / 96
    if scale_ratio != 1.0:
        coords = [int(item*scale_ratio) for item in coords]

after

    if absolute:
        coords = ctrl.client_to_screen(coords)

Relevent Problem

pywinauto.mouse._perform_click_input may also need to be adjusted.

Specifications

• Pywinauto version: 0.6.8
• Python version and bitness: 3.7
• Platform and OS: windows 10

[nixgnef]

In the above case, using select() to substitute click() also works:

            tree_view = app.top_window().child_window(
                control_id=129, class_name="SysTreeView32"
            )
            target_item = tree_view.wrapper_object().get_item(['query','target'])
            target_item.select()

[vasily-v-ryabov]

It looks like the same as #37 (pretty old issue). But need to investigate it. Could be separate issue.

[nixgnef]

@vasily-v-ryabov I had seen #37 , but I still thought it's necessary to open a new one, because I figured out that this issue is caused by the DPI(dots per inch) setting of windows OS, which is not apparently mentioned in #37 .

[airelil]

BTW, the 96 is the default DPI?

[nixgnef]

@airelil 96 is the DPI corresponding to the 100% scale ratio, however, the default scale ratio may not be 100%(in my machine is 125%). In fact, this problem is more complicated than I had realized. Because there are three DPI modes(see the doc from microsoft DPI Awareness Mode) , and for each mode we may need a specified way to get the right coordinates. Nevertheless, it should still be helpful to note the DPI change may cause this issue.

[airelil]

@nixgnef, thanks, that's good article. We've done some work for DPI awareness: https://github.com/pywinauto/pywinauto/blob/51e67932650702d5f3f2d1c7efa91150ac3b2a74/pywinauto/windows/win32functions.py#L752
Seems as it needs to be reviewed again. BTW, the article also mentions new added enum we're missing:

PROCESS_PER_MONITOR_DPI_AWARE_V2 = 3, /* Newer HiDPI type for Windows 10 1607+ */

[nixgnef]

@airelil I've found addtional docs relevent to this issue:
1) DWM scaling
2) High DPI Settings in Windows

This issue may be caused by a setting in which the coordinates got from the API and the coordinates displayed do not match(differ by the scale ratio).
To figure out the exact way to get the right coordinates in different situations is cumbersome, a simple way to handle this issue is to add extra parameters to let the user to decide whether to make an adjustment and the scale ratio to adjust the coordinates。
Finally, people who use pywinauto include me should thank you contributors of this project. :smiley:

[airelil]

So you suggest adding scale parameter to user facing methods, such as: HwndWrapper.click(), double_click and close_click?
https://github.com/pywinauto/pywinauto/blob/1b474df126444ac45729709bb6e9f39fccf57877/pywinauto/controls/hwndwrapper.py#L731

[nixgnef]

@airelil 🤝 Yes.
1) This issue will not occur in the well programmed new windows applications.
2) The comment for this newer added parameter can remind users the existence of this issue and to avoid this issue, users can try to set the scale ratio to 100% for each monitor, or to adjust the scale ratio for each click() to match the corresponding monitor setting.

[airelil]

Yep, we can consider it. Also, I had a bit time today to check this. I noticed that calling SetProcessDpiAwareness with the new PROCESS_PER_MONITOR_DPI_AWARE_V2 (3) might help. You can try to make this change in your pywinauto code too.
However, I couldn't find the value to be documented. The PROCESS_DPI_AWARENESS still has old enum definitions. So I want to try switching to SetProcessDpiAwarenessContext instead, but need to find proper values for DPI_AWARENESS_CONTEXT handle

[nixgnef]

@airelil :handshake: Ok，I will have a try at my leisure.

[laoyur]

okay... it cost me one day to figure out why _listview_item.click() chooses wrong item... :(

[nixgnef]

okay... it cost me one day to figure out why _listview_item.click() chooses wrong item... :(

I also spent a little time to find the real cause，so I opened this issue to remind other users。

[bratao]

Yeah, I have been bitten by this too. Any chance of fixing it on a upcoming version?

[nixgnef]

Yeah, I have been bitten by this too. Any chance of fixing it on a upcoming version?

To handle all situations could be tedious, this issue may be caused by the OS setting, the application setting and the compatibility between the OS and the application.