Create Tell Block for UI Interaction - "UI Browser Lite"

Okay, I know I only just revisited this macro, but the version I posted yesterday is already outdated. It's dead to me. I spit on its grave.

I've unified the interface and I think it's rather nice now.

(The macro has been updated in the first post of this thread.)

4 Likes

I adore self-deprecation. And you are very good at it.

2 Likes

No, I'm woefully bad at it. :stuck_out_tongue_winking_eye:

3 Likes

I liked your post, but not sure I should have done that!

1 Like

This new interface is a work of art, it looks great and such a nicer way of grabbing things. I deleted the previeous three I had that were doing similar things.

It took me a second to figure out that you updated it at the very top of the thread and not in your last post.

Thanks for sharing this, I will certainly be using this often!

1 Like

Thanks! I'll make it clearer in the previous post...

I can't seem to get it to work on buttons in the system settings windows.

image

I am guessing this is the reason and is still the case but just wanted to be sure.

Also just a curios side question if you know the answer. When I search for "Local__TestDelay" it also shows that it is in the Custom Prompt with HTML text though I don't see anything with "Local__TestDelay" in it. Why would Keyboard Maestro also highlight that action?

Yeah, as I recall System Preferences is a bit finicky, and it may be something to do with the fact that the items we want to target are within panes within windows. There may be a way around it, but I'm not aware of one.

It's a big ol' script but they're in there...

Screenshots

Thanks for confirming.

Thanks, that was my bad (I should have checked it in another editor first) I didn't realize my BBEdit search settings were off and didn't have "Wrap around" checked. You saved me for the future.

1 Like

Update:

Added the option to identify the UI element by its AXHelp attribute rather than element number. Useful when apps (e.g. Logic Pro) tend to change their element numbers dynamically, which might otherwise scupper attempts to interact with them via AppleScript.

CleanShot 2025-06-25 at 20.42.44

1 Like

Update:

Another thread drew my attention to a parsing oversight, where element numbers were being erroneously wrapped in quotes. The new fix seems to have solved that now.

2 Likes

Update:

Added options for Get Coordinates that give you coordinates relative to the screen or any corner of the front window:

3 Likes

Perhaps this is a dumb question, but can this script only be used for apps that are scriptable? I tried it with a Java app, to click the hamburger button, but it reported:

Macro Cancelled nv
Execute an AppleScript failed with script
error: text-script:807:821: execution error:
System Events got an error: AppleEvent
handler failed. (-10000). Macro "Generate UI
Element Tell Block” cancelled (while
executing Get Info for Ul Element Under
Mouse).

Yeah it's Applescript only. I think one for Web elements would be doable too but it may be a while before I get round to it.

1 Like

That would be so great.

This isn't as fancy, but it will return JS for clicking the element under the mouse.

1 Like

While not wanting to disagree with @noisneil about his own macro...

The app doesn't have to be scriptable -- the macro uses System Events to generate System Event tell blocks that interact with the app's UI for precisely those times when an app (or feature of an app) isn't scriptable.

But an app does have to have "proper" macOS UI elements for the macro to be of much use -- and most Java apps are, at best, a collection of "proper" windows containing groups of Java UI elements that can't be distinguished. So for the Java image analysis app Fiji, using @noisneil's macro on any button in the toolbar

...gives the same result:

And I suspect that the best you can do with CafeTran will be to generate AS that clicks the single group within that window, not the hamburger itself.

1 Like

Oh sorry, I didn't realise it was an app. I thought it was a website, hence the effort to create a JS version of this in the thread I linked to above.

2 Likes

Luckily, Keyboard Maestro’s identify and click button work in the actual UI. E.g. to set the settings in the Find dialog.

1 Like