How Do I Access Specific Window Elements (UI Scripting)?

Hi Chris,

Your KM Macro is really useful! I'm using KM since few years, improving my workflow for sound editing and mixing. So I'm starting to learn about AppleScript inside KM and my major interest is about how to get the name of buttons, menus, sliders etc. Your tool is really good!

Now I wanted to ask you where I can learn the syntax of Apple Script. For example, I attach a Screnshot with teh inspection of a window. And I want to do the following in this floating window:

  • Select from the menu "entire selection" another option

  • Click on Render button. I know this, but this windows is called "Audio Suite: RX 7 Connect" but I want to hit Render to any window called "Audio Suite: " ---something---. I mean, I can open as AudioSuite another plugin, it still has Render button but the window name would be different. I want an universal script for hitting Render in any window that contains "Audio Suite: "-> is this possible?

Thanks!!

And last...Can I "read" the position of a certain item (a button) and store in a KM variable?

For learning AppleScript in general, see:

Learning AppleScript

If you are new to AppleScript and are interested in learning more, then I'd suggest you take a look at:

  1. AppleScript: Beginner's Tutorial
  2. Script Debugger 7 -- a full IDE for developing AppleScript.
    SD7 now comes with a 20-day free trial, and then will gracefully downgrade to the FREE SD7 Lite (which is still much better than Script Editor).

You can get lots of AppleScript help here in this forum, and in the Script Debugger Forum.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

So what you want to do in this case is called "UI scripting", and it can be a challenge.
If you have much UI scripting to do, then I'd recommend two tools:

UI Browser will definitely make it easier to ID the window elements, and it generates the AppleScript to access a selected element. But it is not cheap.
So, as an alternative, you can use Chris @ccstone's tool with SD7.
I highly recommend Script Debugger 7.

Answer: Probably. If the window title contains "Audio Suite:", then this will be much easier.

Here's an example script I have for closing a specific window:

set scriptResults to "NOT FOUND"
try
  tell application "System Events"
    
    set winList to every window of application process "UserNotificationCenter" whose name of static text 1 is "EvernoteSpotlight quit unexpectedly."
    
    if (winList ≠ {}) then
      set oWin to item 1 of winList
      
      tell oWin
        tell button "Ignore"
          perform action "AXPress"
        end tell
      end tell
      set scriptResults to "EN SL Window Closed"
      display notification scriptResults sound name "tink"
      
    end if
    
  end tell

Maybe that will help you get started.

Yes. If you can identify the window element reference, then you can get or set its value.

Since it is best to start a new Topic when you have a new question/subject, I have moved your post to a new topic.

Oh! Thanks! Good start! I’ll be back in the forum to get more help as soon as I learn the basics and start scripting.

For me the syntax is quite different the other languages I know. Let’s see if I can get it!

Bye!

The AppleScript syntax is terrible -- worst of any of the many languages I have used over the decades.
A much better scripting syntax can be found in JavaScript for Automation (JXA).
If you happen to know JavaScript, then JXA would be a much easier scripting language to learn and use.
If you want to learn JXA, see: JXA Resources

Two downsides to using JXA vs AppleScript:

  1. SD7 does NOT support it, so you are forced to using the lame Apple Script Editor.
  2. There are far fewer examples available for JXA than AppleScript.
    • However, you can often convert an AppleScript to JXA.

Good luck, and let us know how it goes.

Yeah, maybe javascript is more suitable for me. I wrote some cose with Apps Script which I suppose is javascript based; however the important thing is how much information on the web about a certain language.

In just read de Apple Script for Beginner you pointed me and is interesting to learn.

I think the most difficcult part is to discover the name and hierarchy of certain item and the nane of the properties you want to change.

And what about swift?

I have read the Apple Script tutorial for beginners you suggested me yestarday. Really interesting, I think I’ll be able to manage it, it’s plain english language!

So I see UI Browser is really powerful, it worth it 50 bucks. By the way I see Script Debugger 7 has also exploring and inspecting capibilities. Is it possible to inspect UI with SD7?

Yes, indirectly.
It does not work at all like UI Browser, but you can use "System Events" to get a reference to a window, and then view all of the window elements and properties in the SD7 Inspector Panel.

Here are a few SD Video demos to illustrate this:

And for a more complete selection of SD Demos, see:

Here's a simple example of getting UI Elements of Apple Mail

tell application "Mail" to activate

tell application "System Events"
  tell application process "Mail"
    set winList to every window
    set oWin to item 1 of winList
    
    
  end tell
end tell

Wonderful Michael, you gave my all necessary tools!

Alberto!
What you have described is a major pain in the behind! For some reason Avid didn’t make that menu option part of the user storable presets. Drive me nuts.
From memory, other people have written about this in the past and Avid apparently don’t define the audio suite window as Audio Suite. people couldn’t seem to find what it’s called.
Maybe you can?

I would very much appreciated it if you would be happy to share your results?

All the best.

It will take some time, because in these months I’ll have much work to do, but I’ll dig it with Script Debugger and searxhing in forum. And sure I’ll share here!

Bye

Hi Alberto.

I have made something that seems to do the trick. A bit crude but no mater where the audio suite window is, KM will detect the "image" of the Audio Suite window and change it to Clip By Clip.

32%20am

Thank you for your effort. I actually know how to do this with found image. But I'm asking how to do via Apple Scripting in order to be more solid. Actually your macro won't work because the window change all the time. The content change depending of which plugin is charged and all cascade menus can hace a different selection.

I'm working in it,,,,too

Now I am pretty sure that I have a couple of KM macros on my work computer that do:

  1. clicks the necessary popup menus of the frontmost window that contains "Audiosuite" relative to the top left corner; (as I haven't yet figured out how to navigate the sub menus via Apple Script) which I have found to work consistently so far (I'm always having to switch back and forth with the "clip by clip" and "entire selection" options).

  2. I'm pretty sure I have an applescript in KM that clicks the "render" button of any window that contains the name "Audiosuite" with the index 1... So I just hit my trigger and then whatever Audiosuite window that's at the front hits "render"

I'll have a look when I'm back in (hopefully I remember) and try to post them here. Hopefully they can be of use.

I answer to you.

  1. I found this code I tried to adapt to my case but I couldn't
tell application "System Events"
    tell process "Messages"
        set frontmost to true
        if not (exists (1st window whose value of attribute "AXIdentifier" is "MessagesPreferencesWindow")) then
            keystroke "," using command down
            repeat until exists (1st window whose value of attribute "AXIdentifier" is "MessagesPreferencesWindow")
                delay 0.1
            end repeat
        end if
        tell (1st window whose value of attribute "AXIdentifier" is "MessagesPreferencesWindow")
            tell pop up button 4 of group 1
                click
                delay 0.2
                click menu item "Popcorn" of menu 1
            end tell
        end tell
    end tell
end tell
  1. I got it through Apple script with this code: How to modify do something to many window which name star with same words . This last is working!