Hi,
Is there way for Applescript to dynamically pass uielements to KM Maestro for mouse simulation such as control click, command click, drag and drop to another uielement, and so on?
Applescript has only click method, so it’d be great if we can leverage KM capability.
Thanks,
Hey James,
Many things are possible, but it’s not all that easy.
Search Google for AppleScript UI Scripting.
–
Best Regards,
Chris
Please correct me if I’m wrong, but it looks like UI scripting through system events doesn’t have much other than just regular click method. Also many people seem to try to find the solutions to I’m looking. That’s why I posted here if we could use KM to achieve this along with Applescript.
Hey James,
You can actually find the location of UI-Elements via System Events. From there you can click to your heart’s content with KM.
An example:
---------------------------------------------------------------------------------
# Auth: Christopher Stone
# dCre: 2014/10/05 11:02
# dMod: 2014/10/14 17:55
# Appl: Finder & System Events & Keyboard Maestro
# Task: Expand Name column to fit
# Libs: None
# Osax: None
# Tags: @Applescript, @Script, @Finder, @System_Events, @Expand, @Name, @Column
---------------------------------------------------------------------------------
tell application "Finder"
if current view of front window ≠ list view then error "View of front Finder window is not list view!"
end tell
tell application "System Events"
set quit delay to 0
tell application process "Finder"
# set frontmost to true
tell window 1
-------------------------------------------------------------------------
# Strange discrepency for ~/Documents/
-------------------------------------------------------------------------
if image 2 of scroll area 2 of splitter group 1 exists then
tell image 2 of scroll area 2 of splitter group 1
set {xPosition, yPosition} to position
set {xSize, ySize} to size
end tell
else if scroll area 2 of splitter group 1 exists then
----------------------------------------------------------------------
# Toolbar & Sidebar Showing
----------------------------------------------------------------------
tell image 2 of UI element 1 of row 1 of outline 1 of scroll area 2 of splitter group 1
set {xPosition, yPosition} to position
set {xSize, ySize} to size
end tell
else if scroll area 1 of splitter group 1 exists then
----------------------------------------------------------------------
# Sidebar-only Hidden OR Toolbar & Sidebar Hidden
----------------------------------------------------------------------
tell image 2 of UI element 1 of row 1 of outline 1 of scroll area 1 of splitter group 1
set {xPosition, yPosition} to position
set {xSize, ySize} to size
end tell
----------------------------------------------------------------------
end if
set {xPosition, yPosition} to {xPosition + (xSize div 2), yPosition + (ySize div 2)}
end tell
end tell
end tell
tell application "Keyboard Maestro Engine"
try
set value of variable "X" to xPosition
on error
make variable with properties {name:"X", value:xPosition}
end try
try
set value of variable "Y" to yPosition
on error
make variable with properties {name:"Y", value:yPosition}
end try
end tell
---------------------------------------------------------------------------------
The AppleScript is part of a KM macro:
Window { Resize Name Column }
Triggered by any of the following:
The Hot Key ⌥R is pressed
Will execute the following actions:
Execute AppleScript
Path: ~/Library/Scripts/Applications/Finder/Window { Resize Name Column ; KM}.scpt
Move and Double Click
At (X,Y) from the absolute position.
It’s not real fun, but it can be done.
It’s easier if you have UI Browser from pfiddlesoft (about $55.00 US).
–
Best Regards,
Chris
Hi Chris, that’s awesome!
Thank you for the code!
Sorry I’m a newbie here and sorry again if it’s a stupid question. How do you access to system events?
Hey Chris, I want to click on the share button in Finder window toolbar. With UI Browser I got this far:
tell application "System Events"
tell process "Finder"
click button 1 of toolbar 1 of window "UI Browser"
end tell
end tell
How to I modify this to work with ANY Finder window rather then a specific one like this?
Thanks
I'm obviously not Chris, but maybe I can help.
In direct answer to your question, almost always the frontmost window can be referred to as "window 1".
But I think you may have a bigger problem of properly referencing the Finder Share button.
Here is a more generic solution, that does not care where the Share button is located.
Below is just an example written in response to your request. You will need to use as an example and/or change to meet your workflow automation needs.
Please let us know if it meets your needs.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-~~~ VER: 1.0 2020-04-25 ~~~
Requires: KM 8.2.4+ macOS 10.11 (El Capitan)+
(Macro was written & tested using KM 9.0+ on macOS 10.14.5 (Mojave))
Select a Menu Item on Finder's Share Menu [Example].kmmacros
Note: This Macro was uploaded in a DISABLED state. You must enable before it can be triggered.
property ptyScriptName : "Select Menu Item on Finder Share Menu"
property ptyScriptVer : "1.0"
property ptyScriptDate : "2020-04-25"
property ptyScriptAuthor : "JMichaelTX"
activate application "Finder"
tell application "System Events"
tell process "Finder"
set oShareBtn to first button of toolbar 1 of window 1 whose description is "Share"
tell oShareBtn
--- First Part to Avoid Delay of Menu Response ---
-- Menu shows quickly, but then has an unexplained delay of several seconds
ignoring application responses
perform action "AXPress"
end ignoring
------------------------------------------
-- Better to Use KM Insert Text by Typing
------------------------------------------
(*
The below commands usually work, but it is likely
to be faster to use the KM Type Action.
*)
--set menuFound to false
--repeat until menuFound
-- delay 0.1 -- adjust as needed, 0.2 to 1.0 sec
-- try
-- set oMenu to menu 1
-- set menuFound to true
-- -- look at oMenu in Script Debugger to see all menu items
-- end try
--end repeat
--
--tell menu 1
-- set oNotesMenuItem to menu item "Notes"
-- tell oNotesMenuItem to perform action "AXPress"
--end tell
end tell
end tell
end tell
--- Part 2 to Avoid Menu Delay ---
delay 0.5
do shell script "killall System\\ Events"
delay 0.5
Thank you JM, this works. I'm grateful to again be on the receiving end of your generosity 
.
The items "Commented Out" by "--" at the beginning of the lines or by brackets (* text *) and are greyed out are notes and alternative code to try, I understand.
I appreciate the inline notes explaining the sections within the script as they make it much more readable for my level of understanding. Thank you.
As noted there is a significant delay. Not bad and faster is usually better.
You say
-- Better to Use KM Insert Text by Typing.
What text should be inserted where?
This is followed by
(* The below commands usually work,...*)
Are you suggesting substituting these commands for the ones above or in addition to what's above and where are they to occur, where they are now, just prior to the three "end tells"?
"(... but it is likely to be faster to use the KM Type Action.)
Sorry to be so easily confused. Are you saying to use a Type action (insert by typing or pasting I assume) instead up alternative greyed out code and if so what is being inserted by type?
Thank you for suggesting using Script Debugger for alternative menu items. I see this could be used to press many buttons.
The macro I provided shows exactly how to do this.
If you use KM macro to type the name of the menu item, then you don't need the AppleScript code that has been commented-out.