Select GUI Control Point, When UI Browser Doesn't Identify a Scriptable Action

[whitebalance]

Hello,

I'm trying to figure out how to find a GUI control point, assign a left click and hold command that can then be used by a Kensington trackball to move that control point, and release after I press a button in the DaVinci Resolve application, Curves group.

I also may be approaching this wrong. Is there another way to do this? Python, JS Node, Command Line?

Is there a way to "browse" shell commands for an application the way UI browser's identify scriptable UI elements?

Don't bother googling DaVinci Resolve API docs, they are very light.

The first part is relatively straightforward, as initiating the GUI control point is clearly visible in the four GUI inspectors I have access to, as you can see from screenshots below.

1. Initiate GUI control point:

It looks like I could just use an applescript, or even display position and click, to engage the GUI control point by clicking the GUI control radio button within red box number 1 at the bottom. Once that is done, the application creates a second GUI control point inside the red box number 2. If I use a mouse, I can now click, hold, and drag that second GUI control point and move it around, affecting the Curves control in DaVinci Resolve.

2. How can instead create an applescript or command line shell to:

a. Select the first GUI control point (box 1)
b. Identify the second GUI control point (box 2) created after the selection of GUI control point 1.
c. Click and hold the second GUI control point (box 2) so that I can use a Kensington trackball to move it around.
d. Assign a button to release control of the second GUI control point.

You can see Xcode's UI browser provides a performable action for GUI point 1 (box 1):

I can't find an executable action listed in the four UI GUI inspectors for the second GUI control point (box 2), when I click on that second GUI control point:

The whole box is green, does that mean the whole box is treated as one selection, and nothing inside can be scripted?:

Here are two macro UI identifiers generously provided on KM's forum:

[ccstone]

@whitebalance

Is there another way to do this? Python, JS Node, Command Line?

Hey Ernest,

Nyet.

All accessibility goes through Apple's Accessibility API, and it's up to the developer to support it.

Many don't – many more don't do a good or thorough job.

Is there a way to "browse" shell commands for an application

There are few if any shell commands available for applications, unless they support them of course (and few do).

The AppleScript dictionary for any given app can be seen (or seen to not exist) by dropping the app on Apple's Script Editor.app.

You’ve done a good job describing your task, but it's extremely difficult to give advice about UI-Scripting without having the app to tinker with.

You don't know what will work and what won't until you try it, and that's when the Accessibility Inspector can see the UI element.

When it can't see a UI elements you've slipped into the creek without a paddle much less a boat.

I'll look at this a little more when I have more time, and more importantly after I've slept. 

-Chris

[whitebalance]

Alright I'm super close!

Simple image recognition:

https://drive.google.com/file/d/1bp5dZbDc1E6-svI-SRbJcBnGwSfhQFCN/view?usp=sharing

But I cannot drag with the mouse to move the point, once it's selected, even though I have click and hold selected in the action. I tried to also just move, then added a another click and hold action. I tried adding a pause.

Once it selects the image, how do I get it to click and hold, so I can move the GUI control point with a mouse?