I'm trying to use method IUIAutomation::ElementFromPoint in Python using comtypes package. There are many examples how to use it in C++, but not in Python. This simple code reproduces the problem on 64-bit Windows 10 (Python 2.7 32-bit):
import comtypes.client
UIA_dll = comtypes.client.GetModule('UIAutomationCore.dll')
UIA_dll.IUIAutomation().ElementFromPoint(10, 10)
I get the following error:
TypeError: Expected a COM this pointer as first argument
Creating the POINT
structure this way doesn't help as well:
from ctypes import Structure, c_long
class POINT(Structure):
_pack_ = 4
_fields_ = [
('x', c_long),
('y', c_long),
]
point = POINT(10, 10)
UIA_dll.IUIAutomation().ElementFromPoint(point) # raises the same exception
You can reuse existing POINT structure definition directly, like this:
import comtypes
from comtypes import *
from comtypes.client import *
comtypes.client.GetModule('UIAutomationCore.dll')
from comtypes.gen.UIAutomationClient import *
# get IUIAutomation interface
uia = CreateObject(CUIAutomation._reg_clsid_, interface=IUIAutomation)
# import tagPOINT from wintypes
from ctypes.wintypes import tagPOINT
point = tagPOINT(10, 10)
element = uia.ElementFromPoint(point)
rc = element.currentBoundingRectangle # of type ctypes.wintypes.RECT
print("Element bounds left:", rc.left, "right:", rc.right, "top:", rc.top, "bottom:", rc.bottom)
To determine what's the expected type for ElementFromPoint
, you can just go to your python setup directory (for me it was C:\Users\<user>\AppData\Local\Programs\Python\Python36\Lib\site-packages\comtypes\gen
) and check the files in there. It should contains files automatically generated by comtypes, including the one for UIAutomationCore.dll. The interesting file name starts with _944DE083_8FB8_45CF_BCB7_C477ACB2F897 (the COM type lib's GUID).
The file contains this:
COMMETHOD([], HRESULT, 'ElementFromPoint',
( ['in'], tagPOINT, 'pt' ),
This tells you that it expects a tagPOINT
type. And this type is defined a the beginning of the file like this:
from ctypes.wintypes import tagPOINT
It's named tagPOINT because that's how it's defined in original Windows header.