before I post my question, I wanted to mention that this is my first posting here. I found KM last weekend and am super excited about all the opportunities since.
One of my most important use cases in my day job is to click certain menu items on a webpage. While I have found a work around for now, I am wondering if there is a better way to do it. My main problem: There are a view variables on the website that impact where that menu is located.
Initially I thought the menu item could be identified as an image but that did not work at all.
What does mostly work is this: I created four different macros, one for each possible option of the menus location and using the mouse movement and click to manually select the mouse’s correct position. It’s fairly easy to identify the different settings that define where the menu is and so I press the appropriate hotkey accordingly.
However my solution requires that the window is placed correctly and zoom is on the right level. Both turned out to be a bit tricky causing me to redo all the mouse positions multiple times already.
Do you think there is a better solution? I read a little bit about XPath but I cannot program
There could be a few solutions, but first: Which browser do you use? And are the menu items text or embedded images?
My initial idea (because I primarily use Firefox) was to have a macro that sends ⌘+F keystroke; pastes the text of the menu item (assuming the menus are text and not images); types the Esc keystroke; then sends the Return keystroke. But there could potentially be other solutions if you use Safari.
Make sure you you screenshot to the Clipboard (⌃⇧⌘4 and drag across the are to capture) and then Paste into the action's image well, rather than screencapturing directly to file. That solves a lot of resolution problems. It won't solve page-zoom issues -- but you could always either make multiple images, one for each zoom level, or have your macro set the zoom before doing image detection.
Hey both, thank you for your replies, I appreciate the help.
The browser is Firefox. I am not sure what the button technically is. If I right click it, it does not give me the option to save an image so I believe it might be something else they use. The button does include text. I attach images of the menu.
@Nige: It’s exactly how I tried it. Just to make sure, we are both speaking about the action „Move and click at (0,0) from the Center of the Found Image?
Are you able to navigate to the menu with ⌘+F? As in, if you type ⌘+F and then type the menu title, does Firefox select and jump to the text of the menu? If so, then you can have a macro navigate the menus without using the Click at Found Image action.
Also, do you specifically need to click on the arrow next to the menu, or can you just click on the menu item itself?
You are fast CMD+F indeed finds the text. In regards to the triangle: In some cases its enough to click on the text next to it, in other cases i really need the sub menu that appears after clicking on the triangle.
I made some quick progress in the meantime: When I did a screenshot only from the triangle with the surrounding frame, that worked as a first step. Wenn I thando the same action again and use a screenshot of one of the text below, it does not work
Then your options are limited, since Firefox can't be automated via AppleScript -- so no "Front Browser" KM actions.
You should be able to use the "Click at Found Image" action, though. You can use the action's "Go" button to test the image detection and may have to change the slider (moving it to the right allows a "fuzzier" match) along with other options.
It really does depend on the web page as to the settings you'll need. Is it private or something you can point the rest of us to?
Almost looks like the screenshots do not work on the text for some reason? the triangle works just fine but in the upcoming sub menu where there is only text it does not work anymore. Error message is "No unique image found".
Unfortunately I can not link you to the page directly, as its behind a user account.
On the browser: While I have to use Firefox in general, for this particular task I should be able to use Safari if this would be easier? Would have to wait until Monday to try that though.
In the case of using the dropdown arrow, try with the keyboard the following steps:
⌘+F
[the full text of the menu you want to navigate to]
Esc
Tab
Return (or Space)
If those steps reveal the dropdown menu, then see if you can navigate the dropdown list with the arrow keys—if not, you can try ⌘+F again and see if Firefox can find the text in the submenu.
These actions are all very easy to reproduce in Keyboard Maestro. Though it seems like you have a working solution since I've been typing this, I'll post it anyway as an alternative for you or anyone else who might find it helpful.
You may have to add a pause between the two "Click at Found Image" actions, to give the menu time to appear. And remember that "No Unique image..." could mean "no image" or "more than one image".
Once you've popped the menu, see if you can do the selection by keystroke -- most menus can have items selected by typing the first few characters then hitting Return, so you might be able to do eg "Type a Keystroke:c", "Type a Keystroke: Return" to select "contacted". Keystrokes can be more reliable than image detection -- they don't care if your pointer is in exactly the wrong position and has highlighted the item, for one!
The CMD+F in the sub menu did not work, as it always disappears once you put the cursor away unfortunately.
Nige something similar to what you described worked. While I was not able to jump to an item by tipping it, the arrow keys + enter worked. That option is great for another use case I have in mind.
CMD+F indeed finds the text. In regards to the triangle: In some cases its enough to click on the text next to it, in other cases i really need the sub menu that appears after clicking on the triangle.
