Correctly Identifying GUI Elements

Questions & Suggestions
17

I created this macro a while ago to identify UI elements for the purposes of AppleScripting. I position the mouse over the element I'd like to query, hold down ⟨control⟩+⟨option⟩, then tap ⟨command⟩. An AppleScript record is copied to the clipboard containing some pertinent information on the UI element

image
image.png1278×1632 373 KB

AppleScript 
use application "System Events"
use framework "Foundation"
use scripting additions

property NSEvent : a reference to current application's NSEvent
property NSScreen : a reference to current application's NSScreen

-- Screen dimensions
set display to NSDeviceSize ¬
	of deviceDescription() ¬
	of item 1 ¬
	of NSScreen's screens() as record
-- Mouse position relative to bottom-left of screen
set mouseLocation to NSEvent's mouseLocation as any
-- Adjust mouse position coordinates to be relative to top-left of screen
set mouseLocation's y to (the display's height) - (mouseLocation's y)

set obj to click at the mouseLocation's {x, y}

set R to {UIObject:obj ¬
	, UIProperties:obj's properties ¬
	, UIChildren:obj's UI elements ¬
	, UIAttributes:name of obj's attributes ¬
	, UIActions:name of obj's actions}

set the clipboard to ¬
	"tell application \"System Events\" to set R to ¬
" & coerceToString(R)
--------------------------------------------------------------------------
to coerceToString(object)
	local object
	
	try
		set S to object as text
	on error E --> "Can’t make %object% into type text."
		set the text item delimiters to "Can’t make "
		set S to rest of text items of E as text
		
		set the text item delimiters to " into type text."
		set S to text items 1 thru -2 of S as text
	end try
end coerceToString

Identify UI Element.kmmacros (18.6 KB)

6 Likes