Sorry, my bad. I omitted the line use scripting additions from the start of the script. I've made the correction to the original reply post now, and you can download the macro here:
Identify UI Element.kmmacros (18.6 KB)
Apologies for the error.
Thanks, that fixed the problem.
May I suggest one small change to your macro so that the user will know something happened?
I was never intending on publishing the macro for public consumption, but came across this thread and posted it because I thought it might be useful and relevant to the discussion.
My feeling was that a macro of this nature is a bit niche, appealing really only to scripters rather than the KM community at large.
That said, I think I'm going to implement your suggestion for my personal use, as I usually glare at the menu bar's icon to wait until it stops animating. I think your way is a bit more sane.
Blah blah. Done. Macro links updated.
s/scriptures/scripters/ 
1 Like
@CJK, thanks again for this very useful macro/script.
It worked fine on Sierra, but the script seems to hang now that I've upgraded to Mojave.
Running Keyboard Maestro 8.2.4 on macOS 10.14.5.
I also tried running the script from FastScripts, and it hung there as well.
Running the script in SD7, it seems to hang at this statement:
set obj to click at the mouseLocation's {x, y}
Can you confirm this issue on Mojave, or do you have a fix?
Thanks.
I'm afraid I don't have Mojave (I'm bricking it about Catalina).
Below is the latest version of the macro that I use on High Sierra. I reviewed the code today and one thing I did notice is that the previous version didn't import the AppKit framework, which it now does. This might resolve the hang, and I would advise anyone who uses this macro to update it to use the corrected AppleScript.
Identify UI Element.kmmacros (19.6 KB)
AppleScript: Latest version on GitHub
System info: AppleScript version: 2.7 System version: 10.13.6
The handler that retrieves the mouse position is below. To do a quick operational check without having to download a full macro or visit GitHub for the script, then here it is:
on mouseLocation()
on mouseLocation()
script
use framework "Foundation"
use framework "AppKit"
property this : a reference to current application
property parent : this
property NSEvent : a reference to NSEvent of this
property NSScreen : a reference to NSScreen of this
on mouseLocation()
set [list, [number, height]] to ¬
NSScreen's ¬
mainScreen()'s ¬
frame()
tell NSEvent's mouseLocation() to ¬
{its x, height - (its y)}
end mouseLocation
end script
result's mouseLocation()
end mouseLocation
1 Like
I'm looking to create macros that click specific buttons on the UI without using "Click at Found Image", both for stability and share-ability.
Could this help me do that? I can see the script outputting a variable "Result", but I don't know how to make KM click at the found element.
Thanks in advance! This would be a big deal.
If you have a number of macros that you want to create using the app's UI, then AppleScript will provide you with the most capability. To ID the UI elements for AppleScript, the best tool is UI Browser.
If your needs are simpler, then KM does offer a few tools to work the the UI:
Representative Examples (there are more):
App UI Elements
- Insert_Text_by_Typing
- Manipulate_a_Window
- Move_a_Window
- Move_and_Resize_a_Window
- Move_or_Click_Mouse
- Press_a_Button
- Select_or_Show_a_Menu_Item
- Type_a_Keystroke
Web Browser UI Elements
(Offered for "Front Browser", "Safari", and "Chrome")
- Click_Front_Browser_Link
- Find_Image_on_Screen
- Focus_Front_Browser_Field
- Previous_Front_Browser_Tab
- Select_Front_Browser_Field
- Select_Front_Browser_Tab
- Set_Front_Browser_Checkbox
- Set_Front_Browser_Field_to_Text
- Set_Front_Browser_Radio_Button
System UI Elements
- Set_Keyboard_Layout
1 Like
Thanks @JMichaelTX.
I grabbed UI Browser and it's able to see the window position and dimensions of "floating windows", which is one of the problems I'm trying to solve!
I just need a way to tile all those floating windows, i.e. put them next to each other.
Here is a screenshot from UI Browser of a floating window in the app "Live".
In this case the floating window name is "Legend/7 Bass", but we don't know the names of the floating windows ahead of time, and the floating windows may share the same name, so we'd just need to tile all windows in "Live" with the type or subrole "floating window".
To be clear, the windows shouldn’t be resized… just placed next to each other with their current size.
Hey @nrightnour,
Okay, this can get complicated when an app doesn't make uniquely identifying windows simple.
For instance – I see nothing to easily discriminate between Keyboard Maestro Text windows.
Since I don't have Live I'm using KM text windows as a testbed.
Run this 2-3 times to create more than one window.
Display Keyboard Maestro Text Window.kmmacros (2.2 KB)
Then run this script to see how to move them:
tell application "System Events"
tell application process "Keyboard Maestro Engine"
set winCount to count of (windows whose subrole is "AXFloatingWindow")
repeat with winIndex from 1 to winCount
tell (window winIndex whose subrole is "AXFloatingWindow")
set position to {0, 23}
end tell
end repeat
end tell
end tell
Do yourself a favor and use Script Debugger to write your AppleScripts. If you don't buy it it will revert to its free Lite configuration after a demo period – and it still makes Apple's Script Editor.app look sick by comparison.
Do NOT write and test AppleScripts in Keyboard Maestro Execute an AppleScript actions. Use a proper development app to get it right and then move your script to a Keyboard Maestro action.
Keyboard Maestro text windows are identically named and have no ID numbers, so I've had to get creative to identify them and move them individually.
I hope the same technique can be employed with Live.
You'll have to play with your positioning parameters to get this to work with your app, but here's a basic tiling script for Keyboard Maestro text windows.
tell application "System Events"
tell application process "Keyboard Maestro Engine"
set winCount to count of (windows whose subrole is "AXFloatingWindow")
set tileWidth to 0
set tileIncrement to 400
repeat with winIndex from 1 to winCount
tell (window winIndex whose subrole is "AXFloatingWindow")
set size to {400, 200} --> ONLY for TESTING with Keyboard Maestro Text Windows
set position to {tileWidth, 23}
set tileWidth to tileWidth + tileIncrement
end tell
end repeat
end tell
end tell
HTH.
Feel free to ask me questions.
-Chris
2 Likes
Chris, thank you so much. This script is very important to me and I'm finally able to make some headway with this.
The tiling script works when you can resize the windows, but in my case I cannot. I've commented out the line:
set size to {400, 200} --> ONLY for TESTING with Keyboard Maestro Text Windows
I changed the "application process" to "Live", and now it repositions multiple floating windows in Live.
However, it isn't properly reading the window width, so it's only tiling/spacing them according to the the "tileIncrement" variable set by the script.
How do I set tileWidth to what's being displayed here in UIBrowser?
Thanks again.
Nathan
Hey @nrightnour,
Something like this?
tell application "System Events"
tell application process "Keyboard Maestro Engine"
set winCount to count of (windows whose subrole is "AXFloatingWindow")
set floatingWindowSize to size of window 1 whose subrole is "AXFloatingWindow"
set floatingWindowWidth to item 1 of floatingWindowSize
set floatingWindowHeight to item 2 of floatingWindowSize -- Included as an example.
set tileWidth to 0
set tileIncrement to floatingWindowWidth
repeat with winIndex from 1 to winCount
tell (window winIndex whose subrole is "AXFloatingWindow")
set position to {tileWidth, 23}
set tileWidth to tileWidth + tileIncrement
end tell
end repeat
end tell
end tell
-Chris
1 Like
Hey @ccstone, thanks for this!
It's working nearly perfectly, and using Script Debugger 7 I managed to get it across the finish line. (Thanks for giving me the final push I needed to grab that, btw.)
I'd post the code but the forum post editor is giving me issues.
How do I change:
tell application process "Keyboard Maestro Engine"
to "the active application?". I want to use this in multiple apps. Thanks!
Edit: I see it may be difficult to do this using Applescript alone, but I'm okay using a KBM-infused solution if we can pass info about the "active application" to the Applescript.
Hey Nathan,
That's simple enough:
tell application "System Events"
tell (first application process whose frontmost is true)
# STUB
end tell
end tell
It gets trickier when the app you're working with is a background process.
-Chris
2 Likes
One last question @ccstone @JMichaelTX . The script was working very quickly at one point (basically instantly), but now is taking 1-3 seconds per window. How can I troubleshoot this?
Hey Nathan,
It's impossible to guess. Post the code.
Code Block Fencing on the Keyboard Maestro Forum and Paste AppleScript as Markdown Macro
Have you rebooted your system?
-Chris
1 Like
I've rebooted several times yes.
use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions
tell application "System Events"
tell (first application process whose frontmost is true)
set winCount to count of (windows whose subrole is "AXFloatingWindow")
set floatingWindowSize to size of window 1 whose subrole is "AXFloatingWindow"
set floatingWindowWidth to item 1 of floatingWindowSize
set floatingWindowHeight to item 2 of floatingWindowSize -- for future use.
set tileWidth to 300
set tileIncrement to floatingWindowWidth
repeat with winIndex from 1 to winCount
set floatingWindowSize to (size of window winIndex whose subrole is "AXFloatingWindow")
set floatingWindowWidth to item 1 of floatingWindowSize
set floatingWindowHeight to item 2 of floatingWindowSize -- for future use.
set tileIncrement to floatingWindowWidth
tell (window winIndex whose subrole is "AXFloatingWindow")
set position to {tileWidth, 523}
set tileWidth to tileWidth + tileIncrement
end tell
end repeat
end tell
end tell
Hey Nathan,
Good deal.
The first steps of troubleshooting Keyboard Maestro are to quit and restart the engine and to reboot.
Get rid of this cruft.
Script Debugger automatically inserts the code, and it is unnecessary unless you're using AppleScriptObjC or otherwise need to restrict AppleScript to a certain version of macOS and above.
Does your script run slow when executed from Script Debugger?
What version of macOS are you using?
What application are you scripting? (Always tell us this; it is often a vital clue.)
-Chris