Howdy folks, in my continuous pursuit to automate Zoom (an effort in futility some days), and thanks to @ccstone great help, I've figured out how to do certain things with AppleScript that aren't otherwise accesible using Keyboard Maestro's built-in options.
However, I've run across an interesting situation that I'm hoping some of the more AppleScript savvy folks can shed some light on.
Using Script Debugger, the following script to click the "More" button under the participants pane works every time:
ignoring application responses
tell application "System Events" to tell application process "zoom.us"
click button 3 of splitter group 1 of window "Zoom Meeting"
end tell
end ignoring
I add the ignoring application responses because otherwise the script takes 6 seconds to run while it's compiling all of the results, results that I don't need. When I run this script from within Script Debugger (or even Script Editor), it works flawlessly, every time. But when I run this script from within Keyboard Maestro, either as a script file, or a script that Keyboard Maestro itself compiles, it doesn’t do anything. It only works when I take out the "ignoring application responses" and "end ignoring" lines. When I remove those lines, it works just fine in Keyboard Maestro, as a precompiled script file or otherwise.
So ultimately, I guess it's not a big deal to leave those lines out as without them the script works fine...I just want to understand why those lines cause the script to fail when ran from within Keyboard Maestro. I have ran it multiple times, saving the results to clipboard, but the results are always blank. So I'm stumped as the why those particular lines causes the script to fail. Any suggestions are greatly appreciated!
I think the delay is caused by apple's accessibility, so when you use AppleScript it has the delay. I seem to remember some topics about this. I also have experienced this delay as well.
Those 6 seconds of hang are the called-app returning a response to the calling-app. This happens quite frequently with menu bar apps, and some other widgets where accessibility has been implemented poorly. I don't understand all the technical reasons, but these faux pas are a routine pain in the can for folk writing UI-Scripts.
You might try running your script as an asynchronous sub-macro.
But – your best bet will likely be to get the position of the the button you want to click with AppleScript and then use Keyboard Maestro to actually click it.
Something like this:
tell application "System Events"
tell application "System Events"
tell window "Zoom Meeting"
tell splitter group 1
tell button 3
set {X_Pos, Y_Pos} to its position
end tell
end tell
end tell
end tell
end tell
set kmInstance to system attribute "KMINSTANCE"
tell application "Keyboard Maestro Engine"
setvariable "local_X_Pos" instance kmInstance to X_Pos
setvariable "local_Y_Pos" instance kmInstance to Y_Pos
end tell
Thanks Chris for the info. Incidentally, I attempted a similar script with another app to click on a GUI button on its interface, but for some reason it never worked. 🤷🏼
Anyhoo, several of my macros that interface with Zoom use a combination of AppleScript, select menu items, and the found image actions. To be honest, they work quite well and very reliably. The one downside to a few of them is that Zoom has to be recalled to the front before certain steps can be executed. That’s why I’m looking to come up with something that can be executed completely in the background.
For instance, one of my macros is to lower all participant hands. Zoom doesn’t have this as an option in the menu bar items, so KM is not able to call it with a select menu item action. Instead, the only way to do it is to click a dynamic button in the participants pane, which then opens another dynamic menu with that and several other options. Initially I had my macro setup to recall Zoom and then click the “More” button using a found image, then the “lower all hands” menu item also using a found image. This worked, but at the cost of recalling Zoom every time I needed to execute it.
But the script I posted earlier (click button 3 of splitter group 1) clicks on the “More” button quite well, even when Zoom is in the background. At that point, the dynamic menu appears on top of whatever application I am currently in, and the click on found image action sees the “lower all hands” and clicks it just fine. Ultimately I would find a way to have AS click that entry as well, but it’s proven to be a little more elusive than the “More” button, even using UI Browser and Script Debugger.
So technically, my macro works. I just always like to understand everything a little better, and the “ignoring application responses” line is causing unknown issues when it’s used from inside KM. Not a big deal really, since removing it doesn’t appear to cause any issues and the macro works fine without it. It was just odd to see complete opposite results from the same code being executed from Script Debugger and Keyboard Maestro. I’ll just keep tinkering with it. At the end of the day, if I can’t get it to work it’s not the end of the world. The way I had the macros set up before worked just fine. Thanks again!
What happens if you run the script in osascript? Generally that should give the same results as Keyboard Maestro.
This is really happening at a level of AppleScript that is beyond me, so I can only guess that with the ignoring application responses the process is ending immediately, and then somehow System Events is aborting what it is doing.
Yes! That's why I was pleasantly surprised to see that the script was able to click the button without Zoom being at the front.
When run with osascript it fails just like in KM when it has the ignoring application responses lines. When I remove those lines it works just fine. I'm stumped, but that's rather easy to do when it comes to AppleScript
Pretty much what I expected (and means it takes Keyboard Maestro out of the equation as far as the issue is concerned).
What about if you add a delay in the script immediately after the click button line?
But yes, I agree, this is beyond me as far as AppleScript behaviour goes - might be worth asking on an AppleScript forum, especially as you get the behaviour with osascript which is an Apple tool.
Thanks for verifying that it's not an issue with Keyboard Maestro. I know many people are frustrated with Zoom's lack of accessibility features and integration with AppleScript so it shouldn’t surprise me (and doesn't really) that scripting it is so complicated.
I'll try adding that delay after the click the next time I'm on Zoom. As it is, the macro works just fine with the script written the way it is...so I might just leave it alone for now haha.
I've been reading the AppleScript forums for a few weeks, and am just now getting the hang of how to write certain things thanks to @ccstone references he shared with me a few weeks ago. My latest accomplishment is to check the DND status and save it to a variable that I haven't been able to figure out ever since Big Sur changed the menu bar and Notification Center. But I came up with a script that appears to work quite well and is rather simple. So I'm making progress!
I just always like to understand everything a little better, and the “ignoring application responses” line is causing unknown issues when it’s used from inside KM. Not a big deal really, since removing it doesn’t appear to cause any issues and the macro works fine without it. It was just odd to see complete opposite results from the same code being executed from Script Debugger and Keyboard Maestro. I’ll just keep tinkering with it. At the end of the day, if I can’t get it to work it’s not the end of the world. The way I had the macros set up before worked just fine. Thanks again!