UI-Scripting with AppleScript, System Events, and UI Browser

Tips & Tutorials
1

Hey Folks,

Many things on macOS cannot be scripted directly due to a lack of scripting support.

Keyboard Maestro is wonderful tool for automating such things, but sometimes it needs a little help to do its job better.

Enter UI-Scripting which provides access to the accessibility API of macOS through the System Events application.

Here’s an example of UI-Scripting you can run with the Script Editor.app:

tell application "System Events"
   tell application process "Dock"
      tell list 1
         tell UI element "Safari"
            perform action "AXShowMenu"
            
            set diagnosticsList to {¬
               "----- PROPERTIES -----", ¬
               properties, ¬
               "----- UI ELEMENTS -----", ¬
               UI elements, ¬
               "----- ATTRIBUTES -----", ¬
               attributes, ¬
               "----- ACTIONS -----", ¬
               actions, ¬
               "----- END -----"}
            
         end tell
      end tell
   end tell
end tell

** Safari needs to be running!

The script will open Safari’s Dock menu and then provide some diagnostic information for the UI-Element where the diagnostic code is located.

I cheated and used PFiddlesoft’s UI Browser utility to acquire the initial Safari Dock item and then I used my diagnostic construct to move down the tree to where it is now.

----- More to come -----

-Chris

3 Likes
2

This is quite amazing. I am actually curious, what things have you or anyone else made taking leverage of such UI scripting?

3

Hey Nikita,

Search the forum for:

“gui-scripting”
“ui-scripting”
“ui scripting”
“UI Browser”

You should be able to find a number of examples.

-Chris

1 Like
4

Little things like:

activate application "Microsoft Word"
tell application "System Events"
	tell process "Microsoft Word"
		click menu button "Bullets" of group 3 of scroll area 1 of tab group 1 of the front window
	end tell
end tell

Clicks this button:

Found with zero effort thanks to UI Browser.

1 Like
5

This UI Browser thing seems to really be useful. Right now I automate ui presses by km's moving of mouse pointer relative to a window :

Things like that :

Will try to play with UI Browser's free trial though. :slight_smile:

1 Like
6

Hey Folks,

I’m linking to a long post I wrote for the Script Debugger Forum:

Best Practices for GUI Scripting

Some of the discussion beyond my post is worth reading as well.

-Chris

1 Like