Hiya. Without being able to access your particular software, it's hard to know if it's an issue with the macro's function or the scriptability of the app you're working with. I think your notion that the software isn't set up to with scripting in mind is likely to be the stumbling block, but it's hard to know.
In terms of you figuring it out yourself, you can capture the output of the Get Info for UI Element Under Mouse action and see if the element or info you're trying to target is contained within it. If it is, you might be able to parse it out as part of the Parse and Generate Tell Blocks script action.
Fortunately, Keyboard Maestro has plenty of other problem-solving tools, and this forum is blessed with a lot of clever people who like a challenge, so if you do run into issues, there's usually an elegant (or inelegant) alternative solution to AppleScript.
Again it looks like my skills are by far not sufficient to get what you are writing about because I couldnât find the âGet Info for UI Element Under Mouse" action.
Inspired by your answer I tried different things. The element I try to click is a text field but the click doesnât happen since it doesnât show the value selected within the field. like it does with a manual click action.
click text field 33 of window "Untitled"
If I change the script to use âselectâ instead of âclickâ it gives the following result:
text field 33 of window Untitled of application process [app name]
But thatâs just meant as a feedback. Please donât invest any more effort. I can find my workarounds for cases like this.
As you mentioned in your macro, it uses a click when activated which I experienced in all other cases. But if I place my mouse over this kind of text fields it stays like field 1 in the screenshot above.
How do you know it is text field 33? Does the click work for text fields with different numbers? This wouldn't be the first time that a developer has masked the actual field with another to "interesting" effect...
It looks like the UI has a very flat hierarchy. Start in Script Editor with
tell application "System Events"
tell process "myApp" -- replace myApp with your app's name
properties of text field 33 of window "Untitled"
end tell
end tell
That'll get you a list of properties, including value that you can compare to the contents of your fields and position that you can match up with KM's Mouse Display coordinates. You can then try other numbers to get a feel for the field layout.
If the field has a unique value you should be able to:
tell application "System Events"
tell process "myApp" -- your app's name here
every text field of "Untitled" whose value is "myUniqueValue" -- your unique value here
end tell
end tell
I wonder if you could try something other than click here? For instance:
tell application "System Events"
tell process "TargetApplication"
-- Set focus directly on the text field
set value of attribute "AXFocused" of text field 33 of window 1 to true
end tell
end tell
That means the dev's haven't included values for those properties -- missing value is an object that's returned in those situations, not a textual representation of "nothing to see here". Of course, the app framework may not allow (or make it easy) to include such -- is this an Electron app?
Thanks and no, it is not an Electron app. Iâm not allowed to name it here because as I mentioned it is for internal use only. Itâs just a tool for a handful of people around the world so it was therefore likely developed with very little effort and a minimal budget.
Hello Neil (@noisneil) I just saw your change to the MacroâŚ
While I think it is a great enhancement to get an element focused, I think itâs important that this has to be an option as well as the ability to click it.
Is it possible for you to bring back the click functionality and keep the new functionality to focus an element?
Focus only replaces click for fields. For all other elements it defaults to click. This makes sense to me as scripting a field click doesn't appear to do anything useful.
Okay ⌠for editable fields whose contain textual or digit values as input that might be true. Good catch on that.
But whatâs about catching up on fields that are static whose also can deliver a lot of information (Of course if a developer has set them up correctly, so they offer the needed information)? Such fields act like a button in many cases and often a click might also change the GUI because their functionality is like the modern NavigationLink.
Focusing them first to get information about what they are for and then deciding whether to click on it or not is also something really important step in GUI scripting.
If your Macro already supports this behavior then feel free to scratch my feature request. But if not then it might be worth integrating it.
Aside: One thing I've noticed is that combo-boxes (e.g. the font size field/dropdown in TextEdit) don't react to click or focus via AppleScript. Not sure what to do about that.
I can't get the text field part to select, short of passing the position back to KM and clicking from there -- so this will double-click the field assuming TextEdit is frontmost:
AppleScript
tell application "System Events"
tell process "TextEdit"
set thePos to (get position of combo box 1 of toolbar 1 of window 1)
set theX to (item 1 of thePos) + 5
set theY to (item 2 of thePos) + 5
end tell
end tell
tell application "Keyboard Maestro Engine" to do script "<dict>
<key>Action</key>
<string>MoveAndClick</string>
<key>Button</key>
<integer>0</integer>
<key>ClickCount</key>
<integer>2</integer>
<key>DisplayMatches</key>
<false/>
<key>DragHorizontalPosition</key>
<string>0</string>
<key>DragVerticalPosition</key>
<string>0</string>
<key>Fuzz</key>
<integer>15</integer>
<key>HorizontalPositionExpression</key>
<string>" & theX & "</string>
<key>MacroActionType</key>
<string>MouseMoveAndClick</string>
<key>Modifiers</key>
<integer>0</integer>
<key>MouseDrag</key>
<string>None</string>
<key>Relative</key>
<string>Absolute</string>
<key>RelativeCorner</key>
<string>TopLeft</string>
<key>RestoreMouseLocation</key>
<false/>
<key>VerticalPositionExpression</key>
<string>" & theY & "</string>
</dict>
"
The dropdown is easier, it's a button element of the combo box:
tell application "System Events"
tell process "TextEdit"
click button 1 of combo box 1 of toolbar 1 of window 1
end tell
end tell
