Help navigating an app without hotkeys

[skypickle]

I am using a photo editing app called Phocus.The right hand side of the app is a panel with four tabs at the top. Anyone of four functions can be chosen by selecting the appropriate tab: Capture, Browse, Adjust, and Export (see image)

This app has very few hotkeys for this panel and these four tabs can be selected with command 1, command 2, command 3, and command 4. but none of the components within each tab can be selected or adjusted with hotkeys.
Within this tab are headings that can be expanded or collapsed (little triangles) You can see the capture and the Job Info are expanded but the remainders are collapsed. Within the active tab, I would like to navigate the headings with option up and down arrows. I would like to be able to expand and collapse headings with option right left arrows. Within each heading I would like to move between the fields with another hotkey.
The contents of each tab are different. For example, below is the Adjust tab with the exposure setting expanded

Again I would like to navigate the headings with option up and down arrows. I would like to be able to expand and collapse headings with option right left arrows.Within each heading I would like to move between the fields with another hotkey.. so for example under the exposure heading, i would like to move between exposure, contrast, brightness,etc. and then be able to increase or decrease the setting.

I created a macro group named phocus and designated the macros only available in it. As I am new to keyboard maestro, can someone point me in the right direction as to which actions I could use. I tried setting a hotkey option right arrow. I chose the action Interface control -> Move or Click mouse and set the coordinates for. 1140,148. This successfully collapses and expands the exposure heading under the Adjust tab.

Although this works, I would have to make a unique hotkey for every action. This is cumbersome. I would prefer to navigate through the headings with option up/down, regardless of which tab is selected. And expand/collapse each heading with option L/R arrws. Similarly, I would like to navigate and adjust the settings of each heading with the same 4 hotkeys.

[skypickle]

BTW, phoocus is freely downloadable here
https://www.hasselblad.com/my-hasselblad/downloads/#133

[DocOck]

@skypickle

I am using a photo editing app called Phocus.

I did a google and this app does not support AppleScript. However there’s a chance it might work with Keyboard Maestro’s Button action, so you should test that. You may need to resort to making a macro that uses Find Image to (a) find which triangle is open and (b) getting KM to move the mouse to click on the next rightward-pointing arrow below it, then returning the mouse to wherever it was.

I think it’s very possible to accomplish this, but since you’re the only one who can see the whole window, you are going to have to take the lead.

In short, you will create a hotkey triggered by the down arrow which looks for the location of the Phocus window on the screen, then looks for the first down arrow in the left column, then looks for the topmost right arrow directly below it. Then it will click on that arrow. (If there is one.) You could also reverse this logic so that the up arrow moves upwards.

This should work fairly well (ie, reliably.) Since there is a box beside the arrow, I think you should search for a combination image of an arrow with a box, which should reduce false positives.

I think it should be possible to use the left and right arrows to navigate between tabs, but I would start with the up/down arrows first.

[griffman]

I took a look, and there's a chance (though not a great chance) that this action...

...might do what you want. With Phocus running, add that action to a test macro, then click the Button drop-down on the right and choose Phocus from the list. This will populate the Button menu with known buttons in Phocus. Now click that Buttons drop-down again, and Phocus will be at the top, with an expansion arrow. Click that, and you'll see the buttons.

The problem is, you'll see a ton of buttons...

...that's just a tiny snippet of the full list. The problem is there's no way to tell what they do, because very few are well named. You'd have to test them to see which ones might be useful. I don't know that any are, but that's where you'd start.

Each time you change panels, you repeat the process to get a new button list...hopefully it changes, which means the items in the panels have buttons.

-rob.

[DocOck]

@griffman

Each time you change panels, you repeat the process to get a new button list...

That’s only a one time set-up issue. Once he finds out the names for each panel, he can write a program to deal with those names.

The problem with “the button solution” is that I don’t think it solves his problem. He wants the down arrow hotkey to determine which tab is open and then open the next one. I think that requires Find Image.

[skypickle]

thank you for your replies. to make a toy example, I tried to recognize the text in the main window of the application. Then my strategy would be to search for the text 'Adjust' and click that. Well I made a macro triggerred by a hotkey _option down) and for the action I chose Image->OCR screen. I restricted it to a window with title containing Phocus. It looks like this:

I minimized Keyboard MAestro, brought Phocus to active and triggered the macro.
I get a dialog saying 'Macro cancelled. OCR image cannot get target. blablabla.'

I also tried Interface Control -> Press a button and filled in the name 'Adjust'

Error: Press button 'Adjust' failed to find button

[skypickle]

@griffman how did you get that button list?

[griffman]

From my prior reply:

With Phocus running, add that action to a test macro, then click the Button drop-down on the right and choose Phocus from the list. This will populate the Button menu with known buttons in Phocus. Now click that Buttons drop-down again, and Phocus will be at the top, with an expansion arrow. Click that, and you'll see the buttons.

You click the text Button, and a drop down appears. Select Phocus from the list, then wait a second, and hit Button again. The top entry will be Phocus, hover that, and a disclosure triangle shows you the buttons.

-rob.

[fap]

This was new for me too. For others this is an example using Keyboard Maestro Editor itself:

1. Click on the down arrow button:

   

2. Select the application. In this case I use Keyboard Meastro

   

3. Select the down arrow button again

   

4. Select the right arrow after the selected application (In this case Keyboard Meastro)

   

Hope that helps

[Nige_S]

@griffman

The problem is there's no way to tell what they do, because very few are well named.

Those are the AXIdentifier values. You can view them with UIElementInspector (amazingly, still working) -- use the "Download Sample Code" button on Apple's Developer site web page, open the zip file, ignore the code files and drag the (pre-built) app to somewhere more useful.

Run the app, put the pointer over the button you're interested in, note the AXIdentifier value. Here's the pointer over KM's Script Trigger button, and you can see the identifier is _NS:186:

[Steve_E]

Thank you for the link to UIElementInspector. Downloaded and working great.

I am considering an update to Tahoe. Does this still work using Tahoe?

Thanks

[griffman]

@Steve_E

Does this still work using Tahoe?

Yes.

-rob.

[Nige_S]

@DocOck

I did a google and this app does not support AppleScript.

Nope -- but it might play nice with AS and System Events...

<tappity-tappity-tappity>

@skypickle

Again I would like to navigate the headings with option up and down arrows.

You can't "navigate the headings" -- they are either closed or disclosed, you can't select them.

The closest you'll probably get is "open the next in the list, or the first if none are open". You could do that with image detection (see your other thread), but here's an AppleScript way:

Disclose Next Pane.kmmacros (4.8 KB)

It does close panes as it goes, and relies on their reported order being the same as in the UI (which it is in my testing). It can take a couple of seconds, especially as you get lower down the list of panes (or have none open at all) -- but that's the problem with having to iterate through lots of UI elements with System Events.

How you then activate a field in the pane is another question altogether. Since many options require a point-and-click it may be easiest to start with that.

Within each heading I would like to move between the fields with another hotkey.. so for example under the exposure heading, i would like to move between exposure, contrast, brightness,etc

Do you realise that this is already built in for many cases? It isn't brilliant, but you can move between slider and input elements -- from a slider to the next element seems to be ⌃Tab, from an input box to a slider is Tab (add Shift to either to reverse direction).

...and then be able to increase or decrease the setting.

Sliders appear to be ⌃↑ and ⌃↓ adjustable if selected by tabbing from a previous element. Numeric dialog use up- and down-arrows, add shift to x10 the adjustment.

The above makes me wonder what other shortcuts you haven't found yet (and, since none of us use the software, we've no chance of knowing about!). So hit the docs and any Phocus user forums in case there is keyboard navigation for those panes after all...

[skypickle]

thank you for your suggestions. however, control up and control down are hotkeys for mission control. They have no effect in Phocus. Also, tab does not aadvance from a slider. However, if the numerical setting to the right of the slider is selected, tab moves the focus to the next tool

[Nige_S]

@skypickle

thank you for your suggestions. however, control up and control down are hotkeys for mission control. They have no effect in Phocus.

Then set your own hot keys. Also, I typoed in the text -- if you look at the actual macro you'll see they're ⌥Up Arrow and ⌥Down Arrow, as per your OP.

Also, tab does not aadvance from a slider.

I even said as much:

@Nige_S

...from a slider to the next element seems to be ⌃Tab...

It's activating any UI element in the first place, so you can Tab or ⌃Tab between them, that is going to be the problem. Phocus looks to be a port of a Windows app and doesn't follow normal macOS UI conventions -- in other apps those sections would each be a group containing UI elements, but here they are contained by the window and when you disclose them their elements become elements of window too.