How to get last selected menu item

I'd like to build a macro that wil let me select a menu item via the "Select Menu by Name" action, or just by clicking a menu, then store that menu item as a global variable, and the next time i run the macro with some different condition (lets say shift held), just select that menu item.

The conditionals and other stuff i can figure out, but I can't seem to figure out how to store a selected menu item.

I see there is a token %SelectMenuByNameText%, which from the description says "The Text entered in a Select Menu by Name" action, but i can't for the life of me make the token do anything.

There's no wiki entry, does anyone know how to use that token? Or another idea to maybe access the last menu item that was selected in some way to store it to run as a token in the "Select or show menu item" action.

There are some threads here that make it seem that's either impossible, or only doable via AppleScript or perhaps Javascript. Links:

Hope that helps;

-rob.

1 Like

I can solve this request in six actions, plus the one additional action for the second macro which recalls the saved data. But I won't post my solution here for two reasons:

  1. You may not like my solution, since it's only 99% reliable. But let me know if you like it.
  2. Four of the six actions in my solution won't work for you unless you manually create the actions yourself, because they contain images, and any images I create won't work for you, because you have a different monitor size/darkmode/scaling/GUI/DPI than I do.

So what I'll do is describe my solution, and if you want, I can guide you through creating the solution yourself.

Let's say, just for example, that the app your user is using with your macro is the Safari app. (Most apps should work, but some apps contain graphic symbols in their menu names, and those apps may require a little arithmetic to get to work.) Here is how one example Safari menu looks.

If you look at the above image, you will see that the selected menu title ("File") is visually distinct from the non-selected menu titles, and that the menu item which the user is hovering his mouse over ("New Tab") is also visually distinct. We will take advantage of these facts to solve the problem.

The solution will have six actions. There will be two groups of three actions that are almost identical.

The first action will use the FindImage action to search for the location in the menu bar of the left side of the highlighted menu title. It will store the found location in a variable.

The second action will use another FindImage action to search for the location in the menu bar of the right side of the highlighted menu title.

The third action will use the OCR action, specifying Apple OCR as the reader, to read the text in that highlighted box and save the result in a global variable called MenuTitle.

The 4th, 5th and 6th actions will be identical, but will instead search for the blue bar underneath the Menu Bar. For this action, you will ignore the rightmost centimetre of the blue box, which always contains a shortcut, and depending on the app, you may have to ignore the first few millimetres of the blue box, because some apps reserve that area for graphics or checkboxes. The resulting text will be saved in a variable called MenuItem.

With all that done, you will have a second macro that the user triggers (as you say, maybe with SHIFT held) that does this:

image

There you go. Since Apple OCR is well over 99% reliable, this will work 99% of the time. Also, FindImage can be made to work correctly at about the same rate (99%) with correct usage.

I suppose if I were to write this macro, I would add a few extra lines of error handling, such as if the images are not found. But that's just fluff, while the main code is simply six actions, as described above.

1 Like

Thank you! I can see how this would work and I'll give it a shot when i have some free time, things might get complicated if the menu item is in a submenu (which some of the ones i'm thinking would be used for this are) but I appreciate your idea!

Yes, that would complicate matters, but since submenus always move from left to right, and since FindImage can detect multiple images in order from left to right, you could still use FindImage to find each submenu and then store each submenu in multiple lines a single variable.

Then in your second macro, you could have a Switch statement that performs a different Select Menu action depending on how many lines you have stored.

It's entirely doable.

1 Like

I'm curious as to where you saw this. I've searched both the KM forum and the wiki without success. Can you point me in the right direction please? Thanks!

I frequently use the "Insert Token by name" action to add tokens, and i came across it there. It's great for quickly finding tokens, the default shortcut is ctrl-command-T, ctrl-command-A is the same for Actions.

1 Like

Yes, I'm familiar with those commands (plus the others for variables, functions and ICU dates) so thanks - I'll have a look there and see if I can make sense of it (but don't hold your breath) :wink:

1 Like

I figured a power user like yourself would be, but I try to never make assumptions! I'm making good progress on @Airy 's suggested method but would be interesting to see if that token did anything.

1 Like

Edit:

Nevermind! I figured it out, the OCR was adding a newline character at the end, when i trimmed whitespace it works.

My Select Menu action doesn't seem to be processing the text tokens. I verified the titles and they are correct but i'm getting an error when i run the macro and it's saying

2024-04-25 14:56:28 Select Menu Item failed to find target menu item %Variable%MenuTitle%➤%Variable%SubMenuTitle%➤%Variable%MenuItem%. Macro “Get Menu Item” cancelled (while executing Select “%Variable%MenuItem%” in the Menu “%Variable%MenuTitle%” in the Front Application).

image

And all variables have appropriate values:
image

As far as I can tell, it never has a value. I tried displaying the token (nothing), filtering the token with Process Token (nothing), counting the characters in the token (it just returns the character count in the actual token string). @peternlewis, how is this token used/usable?

-rob.

1 Like

Really appreciate your idea, i was able to build it and it's totally functional now.

A few things to note in case someone else wants to try to recreate this:

  • For submenus, I used the presence of this icon, in it's selected state to determine if a submenu was active, and the width of the OCR area for finding the submenu title.

  • As you mentioned, I was having some initial trouble determining the proper OCR area to exclude the keyboard shortcut displayed in the menu. I found a nice way around this by using the MOUSEX()-MenuItemLeft.x [the top left of the found image location of the selected menu item] for the width of the OCR Area., as long as the mouse is between the keyboard shortcut and the text, the OCR will only read the menu item.

  • I had to enable "reduce transparency" in the accessibility menu to keep the menu found images consistent since otherwise they would vary based on my background image and the location on the menu bar.

Overall, i've tested it in a bunch of situations and it's working very nicely so far. Thanks for your help!

1 Like

I'm very glad (and I'm also very impressed) that you worked on this puzzle and solved it. Submenus were clearly a little complexity that I didn't fully anticipate, but you solved it with aplomb.

This puzzle is a wonderful example of how to use KM's amazing Find Image action with KM's even more amazing OCR action using Apple's OCR engine. I love both of these features so much - and it's nice when they work together like this.

The code you wrote is quite similar to the code I had envisioned.

The SelectMenuByNameText is actually not implemented, oddly. I'll get that fixed for the next version. It's all coded in the current version except apparently I forgot the actual token.

5 Likes