Create Tell Block for UI Interaction - "UI Browser Lite"

It does, would it be helpful to send an example tell block from UI browser alongside the return of your macro?

Yes, why not. Could you also show me the output of the very first action in the macro too.

Applescript From UI Browser:

click button "Record Enable" of group "Normal Transport Buttons" of group "Transport View Cluster" of window "Edit: TEST-001"

Tell Block from Macro

activate application "Pro Tools"
tell application "System Events"
tell process "Pro Tools"
click window "Edit: TEST-001"
end tell
end tell
Output from first action

UI element:window Edit: TEST-001 of application process Pro Tools, minimum value:missing value, orientation:missing value, position:2, 27, class:window, accessibility description:missing value, role description:standard window, focused:false, title:Edit: TEST-001, size:2555, 1004, help:missing value, entire contents:, enabled:missing value, maximum value:missing value, role:AXWindow, value:missing value, subrole:AXStandardWindow, selected:missing value, name:Edit: TEST-001, description:standard window, AXFocused:false, AXFullScreen:false, AXTitle:Edit: TEST-001, AXChildrenInNavigationOrder:System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, System Events, AXFrame:2, 27, 2557, 1031, AXPosition:2, 27, AXGrowArea:missing value, AXMinimizeButton:button 17 of window Edit: TEST-001 of application process Pro Tools, AXDocument:file:///Volumes/Work%20Audio%20EM//TEST-001/TEST-001.ptx, AXSections:SectionUniqueID:AXContent, SectionDescription:Content, SectionUniqueID:AXContentNavigator, SectionDescription:Content Navigator, AXCloseButton:button 15 of window Edit: TEST-001 of application process Pro Tools, AXMain:true, AXActivationPoint:72, 41, AXFullScreenButton:button 16 of window Edit: TEST-001 of application process Pro Tools, AXProxy:image TEST-001.ptx of window Edit: TEST-001 of application process Pro Tools, AXDefaultButton:missing value, AXMinimized:false, AXChildren:group Edit Mode Cluster of window Edit: TEST-001 of application process Pro Tools, group Zoom Quadrant Cluster of window Edit: TEST-001 of application process Pro Tools, group Cursor Tool Cluster of window Edit: TEST-001 of application process Pro Tools, group Counter Display Cluster of window Edit: TEST-001 of application process Pro Tools, group Grid/Nudge Cluster of window Edit: TEST-001 of application process Pro Tools, group Quantize Cluster of window Edit: TEST-001 of application process Pro Tools, group Transport View Cluster of window Edit: TEST-001 of application process Pro Tools, button ToolbarOptionsMenu of window Edit: TEST-001 of application process Pro Tools, table Track List of window Edit: TEST-001 of application process Pro Tools, static text TRACKS of window Edit: TEST-001 of application process Pro Tools, button Track List pop-up of window Edit: TEST-001 of application process Pro Tools, table Group List of window Edit: TEST-001 of application process Pro Tools, static text GROUPS of window Edit: TEST-001 of application process Pro Tools, button Group List pop-up of window Edit: TEST-001 of application process Pro Tools, button Groups Keyboard Focus of window Edit: TEST-001 of application process Pro Tools, radio button MIDI EDITOR 1 of 3 of window Edit: TEST-001 of application process Pro Tools, radio button MELODYNE 2 of 3 of window Edit: TEST-001 of application process Pro Tools, radio button CLIP EFFECTS 3 of 3 of window Edit: TEST-001 of application process Pro Tools, button Linearity Display Mode - Samples or Ticks of window Edit: TEST-001 of application process Pro Tools, button Edit Window View selector of window Edit: TEST-001 of application process Pro Tools, static text 3 of window Edit: TEST-001 of application process Pro Tools, button Add Marker/Memory Location of window Edit: TEST-001 of application process Pro Tools, static text Selected of window Edit: TEST-001 of application process Pro Tools, table Markers of window Edit: TEST-001 of application process Pro Tools, button Show/Hide SessionView of window Edit: TEST-001 of application process Pro Tools, button Edit Keyboard Focus of window Edit: TEST-001 of application process Pro Tools, button Audio Zoom In of window Edit: TEST-001 of application process Pro Tools, button MIDI Zoom In of window Edit: TEST-001 of application process Pro Tools, slider Slider of window Edit: TEST-001 of application process Pro Tools, button Track List Show/Hide of window Edit: TEST-001 of application process Pro Tools, button Show/Hide Docked MIDI Editor of window Edit: TEST-001 of application process Pro Tools, static text 5 of window Edit: TEST-001 of application process Pro Tools, button Show/Hide Clip List of window Edit: TEST-001 of application process Pro Tools, button 15 of window Edit: TEST-001 of application process Pro Tools, button 16 of window Edit: TEST-001 of application process Pro Tools, button 17 of window Edit: TEST-001 of application process Pro Tools, image TEST-001.ptx of window Edit: TEST-001 of application process Pro Tools, static text Edit: TEST-001 of window Edit: TEST-001 of application process Pro Tools, AXRole:AXWindow, AXParent:application process Pro Tools, AXTitleUIElement:static text Edit: TEST-001 of window Edit: TEST-001 of application process Pro Tools, AXCancelButton:missing value, AXModal:false, AXSubrole:AXStandardWindow, AXZoomButton:button 16 of window Edit: TEST-001 of application process Pro Tools, AXRoleDescription:standard window, AXSize:2555, 1004, AXToolbarButton:missing value, _AXActions:AXRaise

Can you use UI Browser's View Report button to show me the full element tree?

PATH:
application "Pro Tools"
	standard window "Edit: test 00002" (window 1)
		 "Transport View Cluster" (group 7)
			 "Normal Transport Buttons" (group 1)
				button "Record Enable" (button 8)

ACTIONS:
press
show menu

ATTRIBUTES (long values are truncated to 60 characters):
children
	type: (null)
	value: ---
	modifiable: no
label
	type: string
	value: "Record Enable"
	modifiable: no
enabled
	type: Boolean
	value: true
	modifiable: no
keyboard focused
	type: Boolean
	value: false
	modifiable: no
frame
	type: rect
	value: {{1152, 68}, {33, 33}} x, y, width, height
	modifiable: no
help
	type: string
	value: "Record Enable"
	modifiable: no
identifier
	type: string
	value: "1382376224"
	modifiable: no
parent
	type: UIElement
	value:  "Normal Transport Buttons"
	modifiable: no
position
	type: point
	value: {1152, 68} x, y
	modifiable: no
AXReplaceRangeWithText
	type: (null)
	value: (null)
	modifiable: parameterized
role
	type: string
	value: AXButton
	modifiable: no
type
	type: string
	value: "button"
	modifiable: no
selected children
	type: (null)
	value: ---
	modifiable: no
size
	type: size
	value: {33, 33} width, height
	modifiable: no
subrole
	type: string
	value: toggle
	modifiable: no
title
	type: string
	value: "Record Enable"
	modifiable: no
title UIelement
	type: (null)
	value: ---
	modifiable: no
top level UIelement
	type: (null)
	value: ---
	modifiable: no
value
	type: string
	value: "Record Button"
	modifiable: no
visible children
	type: (null)
	value: ---
	modifiable: no
window
	type: (null)
	value: ---
	modifiable: no

NOTIFICATIONS:
announcement requested
created
element busy state changed
focused UI element changed
help tag created
layout changed
moved
resized
row collapsed
row count changed
row expanded
selected cells changed
selected children changed
selected children moved
selected columns changed
selected rows changed
selected text changed
title changed
UI element destroyed
units changed
value changed

I've tried my best, but without actually having PT to test, I'm at a bit of a dead-end I'm afraid.

Perhaps someone else can help. In the meantime, I think this macro is good for most uses, but not heavily nested UI elements, which fortunately seem to be comparatively rare.

No worries, appreciate you looking at it! :slight_smile:

1 Like

I noticed something that maybe could be fixed:
It seems that the only option is to have the block with the "Names" instead of "Index".
It would be good if we could have both options to pick from.

For example I was using it for a Logic Pro macro and it gave me the name of the window "Untitled - Tracks", but the macro will be used for any project, so that will change.
I thought that it would be "window 1", but it was in fact "window 2", but I only found out by using UI Browser itself.

If this is something you can include, it would make it even more complete, in my opinion.

As much as that might be useful for a certain scenario, it's equally useful to reference the window title in others.

I've covered this particular thing above:

1 Like

Yes, I was thinking of a solution for both options, not a "either or" thing.
But I see what you did with the other post and the variable.
Thanks for sharing.

Hey @noisneil I found a bug with the macro. Not sure if this is easy to fix or even possible at all , but here it goes:

On a save as window I'm getting inconsistencies with some elements.
For example with UI Browser I get this:

You see how Where: is surrounded by quotes?
If I use the following in Script Debugger, it compiles and it runs:
SCR-20240708-rsrw

With your macro I get this:

SCR-20240708-rsma

And I get this error in script debugger:

This seems to be the only issue I've seen with the macro so far.
Hope it helps.

I don't have CCC, so can you run this macro with the element in question under the mouse and post its output?

Get UI Element Info.kmmacros (22 KB)

Macro screenshot

This is not limited to CCC.
I experienced it in different places. This is that dropdown menu that lets you pick the destination folder when you are saving something.
For example, go to KM and pick the Export All Macros as Folder and then use your macro when you hover over the dropdown menu:

image

In Logic:
image

Using this macro you shared:

LOGIC

UI element:pop up button Where: of window Save of application process Logic Pro X, minimum value:missing value, orientation:missing value, position:543, 131, class:pop up button, accessibility description:missing value, role description:pop up button, focused:false, title:Where:, size:223, 25, help:missing value, entire contents:, enabled:true, maximum value:missing value, role:AXPopUpButton, value:Logic, subrole:missing value, selected:missing value, name:Where:, description:pop up button, AXIdentifier:where popup, AXEnabled:true, AXParent:window Save of application process Logic Pro X, AXSize:223, 25, AXFocused:false, AXChildren:menu Where: of pop up button Where: of window Save of application process Logic Pro X, AXRole:AXPopUpButton, AXTopLevelUIElement:missing value, AXHelp:missing value, AXPosition:543, 131, AXValue:Logic, AXWindow:missing value, AXRoleDescription:pop up button, AXTitle:Where:, AXFrame:543, 131, 766, 156, _AXActions:AXShowMenu, AXPress

CCC:

UI element:pop up button Where: of sheet 1 of window CCC of application process Carbon Copy Cloner, minimum value:missing value, orientation:missing value, position:543, 187, class:pop up button, accessibility description:missing value, role description:pop up button, focused:false, title:Where:, size:223, 25, help:missing value, entire contents:, enabled:true, maximum value:missing value, role:AXPopUpButton, value:Carbon Copy Cloner, subrole:missing value, selected:missing value, name:Where:, description:pop up button, AXIdentifier:where popup, AXEnabled:true, AXParent:sheet 1 of window CCC of application process Carbon Copy Cloner, AXSize:223, 25, AXFocused:false, AXChildren:menu Where: of pop up button Where: of sheet 1 of window CCC of application process Carbon Copy Cloner, AXRole:AXPopUpButton, AXTopLevelUIElement:missing value, AXHelp:missing value, AXPosition:543, 187, AXValue:Carbon Copy Cloner, AXWindow:missing value, AXRoleDescription:pop up button, AXTitle:Where:, AXFrame:543, 187, 766, 212, _AXActions:AXShowMenu, AXPress

Ah, I can see the omission:

I just needed to add pop up button to the element types in the hierarchy formatting section:

    split("pop up button|button|checkbox|radio button|menu item|slider|text field|static text|group", uiElementsArr, "|")
}{

I've updated the macro above.