Select a GUI Location by Name, and Click on It

Questions & Suggestions
#1

What actions can I use to select a GUI location by its name, without using the more fragile screen position. Is it Text as variable, OCR, what's the best way to get Keyboard Maestro to read on screen text and use that reading as a way to either select that spot, or move the cursor to the spot and click to select.

Here is the screen:

Screen Shot 2021-01-13 at 12.57.16 PM
Screen Shot 2021-01-13 at 12.57.16 PM1920×1200 404 KB

So just to the right of the image of the woman, there is a a square with a name above it called "Safety". That will always be its name, but its position may change. In order to manually select it, I move the mouse to that position and click.

Thanks!

#2

I am able to do this "Click at Found Image" but the image must be the right size, which could change. Is there a "Click at Found Text" or "Click at Found OCR variable" set of actions?

#3

Hey Ernest,

Welcome to the forum!   :smile:

Unfortunately there are no actions that do what you want.

If you run my window analysis tool, you'll get an idea of what UI elements can be accessed.

Many of these heavy-gui graphics apps use non-standard components and have very little accessibility.

-Chris

#4

Thank you for the script. It will save time.

By any chance, do you know if you have 2 identical xKeys 80 key controllers, you can get Keyboard Maestro to recognize them as two different devices? Right now it sees them both as the same device.

#5

No idea...

#6

@peternlewis ⇢ Can you answer his question in post #4?

-Chris

#7

No.

As far as Keyboard Maestro is concerned the two devices are identical, and a button press on one will be identical to a button press on the other.

#8

As a little more than a matter of interest can you post what UI elements the analysis tool reports?

(I suspect a UI as complex as this is painted directly by the app. If that’s the case then UI Scripting won’t get you far. If it is composed of UI elements that’s much easier/better to script.)

#9

Not really. It depends upon what System Events sees.

Look here, and you'll get an idea:

-Chris

1 Like
#10

Hey guys, thanks for the macros to identify GUI elements. I posted another question in the main "questions" forum.

Trying to figure out how to select a GUI control point that doesn't seem to appear in the GUI UI browsers: