HI all, best usage suggestions please. Location clicking in an app window


What I need to create is...

Using a Streamdeck, click a button, that triggers the change to the Cuemix App window. Clicks the "in" then the 48V button, then clicks and drags to set the volume, which is a circular path.
Then trips back to Another App, Digital Performer where we started.

I see the notation on x,y coordinate mouse clicks but it also says that’s not the best way... suggestions please.

Clicking the mouse button by coordinates can work well as long as the coordinates remain the same.

I have no experience with the App you are using, but if the window always opens the same size, you can click relative to one corner of it. So, even if you move the window on screen the mouse will still click in the correct place.

I use this approach successfully with the inbuilt Mac App, Image Capture to set various settings that can only be done by clicking on buttons etc.

If the window is sometimes a different size you might still find that the buttons are in the same relative place compared with one corner of the window (Image Capture works this way).

Since Keyboard Maestro can move and resize App windows, you can also make Actions at the start of your Macro to make sure the window is the right size to get the coordinates to work.

You might also want to put Pause Actions between your various click mouse Actions, to give the System time to catch up. A pause of 0.2 seconds is usually enough. You have to experiment.

If you can't get the coordinate approach to work reliably, then there is always the Click at Found Image Action.

One other thing - any Macro that involves moving and clicking the mouse around the screen will fail if you try and manually move or click the mouse while it is running. It's why (for me anyway) this kind of solution is usually the one of last resort if you can't set settings by automating menu items or some other way. It can work very well but just something to be aware of.

If there's any other way to do this bit -- arrow keys, vertical or horizontal drag, etc -- that'll be much easier. Otherwise you can do a circular(ish) path with a click-and-hold action, a series of move mouse actions, then a button release.

But you need to define your arc! You can get the centre point as @Zabobon describes, using coordinates directly or getting them Image Detect. But you'll also need to decide the direction to "draw" (I assume you want to be able to increase and reduce the volume), the degrees of arc required, and possibly the start point (if the software insists on you starting at the end of the dB meter...). Once you got all that it's "just maths" to find the points your mouse movements will need to start, move through, and stop at.

1 Like

It might not be a circular path (which would be hard to do manually even). I suspect that once the volume knob is selected a drag straight up or down will turn it. Or even better as @Nige_S is saying, arrow keys.

There is a click in for the Numeric variable.
Thank you for your thoughts on that part, your input could come in handy for Panning in DP itself.
But that wasn’t the bigger issue, brought to my attention by the Mouse location FAQ. The bigger issue was Mouse click location, which the faq said there where better ways to do it, that had less variables which could add to failure moving forward. But it doesn’t mention what the better ways are. ???

I have it working with X,y at the moment. But I was hoping to find the other techniques.

Sorry, no idea what FAQ you mean. Have you a link, in case there's some context that helps?

You're (almost) always working with (x,y) when mouse-clicking, even when that's hidden from you by something like "Click at centre of found image...". So you need to find the most reliable way of determining that click-point -- and that will depend on the situation. @Zabobon's outlined some methods, above, like "by absolute screen position" or "by position relative to another, easily found, piece of the window" and even "this far right and down from an image I can find". There's no "best", you'll have to find what works for you and the app in question.

The "better ways than mouse clicks" usually depend on better ways of interacting with the app -- menu items, text boxes, AppleScript dictionaries, etc -- rather than using inherently fragile UI interaction.

I haven't used a Motu interface in a long time, but have you looked into saving your desired settings as a device preset and creating a macro to recall that preset rather than manually changing the settings? Again, i'm not familiar with the app, but just another avenue to consider!

" How can I get the mouse coordinates on the screen or in a window?

If possible, you should avoid using mouse click actions. They tend to be very fragile, easily broken by subtle changes to the system or applications. And they also require the screen to be in the expected state, so you usually need to add a Pause action before them to ensure the item they are clicking on is where it is expected to be.

That said, you can use the Mouse Display window by choosing the Window ➤ Mouse Display menu which will let you see the mouse coordinates relative to the window or main screen. Click the lock button to lock the display after a few seconds, and then put your mouse over the desired location. You can then change the relative corner, and click the Clipboard buttons to copy the coordinates.

Alternatively, you can simply record the click. Turn on recording in Keyboard Maestro, go to where you want to be, wait a couple seconds for the screen to be stabilised so the click will be relative to the front window, and click. Turn off recording, delete any extraneously recorded actions and you have your coordinates. Immediately after recording you can adjust the relative corner of the window or screen, and the recorded coordinates will adjust to match.

Remember that offsets are always to positive to the right and positive down, so if you are making a mouse click up from bottom edge of a window or screen, or left from the right edge of a window or screen, you will need to use negative coordinates."

Thanks ,in this case... Its Cuemix for the Ultralite...
I will be moving on to more specific for DP in the near future.

that seems better yes , thank you