Mouse click on highlighted text

Questions & Suggestions
1

I am looking for a way to click on text highlighted as the result of a matched search in a text editor or browser. E.g. "test" is entered in the search window, the matched text is highlighted and I want to execute a mouse click on that text. I think I need a macro like this one mentioned here but wasn't sure how it was resolved: Move Mouse to Highlighted Text?. It made mention of the "click on found image" action but this doesn't involve an image but rather variable text. Does anyone have suggestions?

The complete workflow is to search for a keyword in a browser window, then when I find the hyperlink containing that keyword to be able to click on it with a keyboard command rather than the mouse.

2

There is a chance that the app you are using provides AppleScript with enough information to solve this completely, but I'm not one of the people who can solve tough problems with AppleScript. If I'm going to be able to solve this using other means, then I need some clarifications.

(1) Which app are you doing the "search" in? Safari? You didn't name the app. The solution may depend on the app. Some apps support AppleScript, and some don't. I know you want this to work with all the apps in the world, but that may not be possible.
(2) When you say "when I find the hyperlink" that's ambiguous. Do you mean you want the macro to keep clicking "Find" on its own and recognize when it has found a link, or do you mean you will keep clicking "Find" until you spot a link, and then the macro can take it from there?
(3) What happens if the "search string" contains two links inside the search string? This could happen! Which link do you want clicked on in that special case?

I actually may have a solution, a simple one, but it may work only 95% of the time. And I'm not sure if 95% is good enough for you. If you answer #1 above, I may be able to give you my solution.

3

A good script is certainly better... but if the highlighted text contains all the necessary information, it might also work like this.

  • copy highlighted text
  • go to the address bar of the browser and paste
  • "Enter"
4

The problem is, it's only highlighted, it's not selected. That's a big problem that I don't know how to solve yet.

5

You're right, it works in a text editor. But not in a browser. And since there is no way to copy the text or move the mouse to the highlighted text ... :man_shrugging:

6

It doesn't work in a text editor for me. For me, it's the same problem there too.

But I specialize in unsolvable problems, and I may have a solution that "moves the mouse to the highlighted text" 95% of the time, if he tells me which app he wants it to work in, and if I have that app.

7

@Airy, @Frankb, Appreciate your thoughts. I'd hope to use this mainly in Mozilla Firefox and in Safari too. I actually don't think I would need this in a text editor like TextEdit, Sublime, VS Code, etc since there is a way to advance the cursor with a keyboard command when a search term is found in the text. The main focus is the browser use-case.

As to @Airy 's questions 2) and 3) from above, the idea was to manually repeat the search for the term until I see it in hypertext. Of course a more sophisticated way that targets a search term only if it is part of a hypertext would be nice to have. Once found, I could invoke the macro to move the mouse pointer to that highlighted location and emulate a mouse click.

95% success is a good start so, yes, that would be fine.

9

Actually, yes. You can search for a word with ⌘f and then continue with ⌘g, the word can be copied. The problem, however, is that this does not necessarily copy the entire link.

10

I can't replicate what you are describing. Based on my tests, the Copy action (or Copy key) doesn't work until text is selected first, and neither ⌘f nor ⌘g select any text. If I saw your macro that can do this, I'd be grateful, because I can't do it.

I googled this issue, and found that Firefox and Chrome need addons to "copy link", and Safari has a way to do it, but instead of using ⌘c, you have to move the mouse over the link, then right click over the link, then use the mouse to click on "Copy Link." And even then Safari copies only the link, not the Display text for that link.

I'm not always right, so I appeal to any of the real wizards on this site to correct me. However I mentioned that I had a method that should work 95% of the time, and I still believe, even with this problem, that my method will still generally work well. So if the wizards can't help you, and you become desperate, let me know. (Yup, I just tested my idea. It works. But as I said, it may not work 100% of the time.)

11

Perhaps we have a misunderstanding :slightly_smiling_face: I'm talking about something very simple. Maybe it only works with Vivaldi. I search for "move mouse" a part of the link is highlighted and I can copy it. But of course this is not the link.

However, when I search for the whole link and copy it, the BTT Clipboard Manager shows me the whole link, which I can paste somewhere. But that is useless. Nobody wants to search for the exact text of the link. I don't know...

image
image2822×888 313 KB

12

I'm sure we do have a misunderstanding. I am trying to solve (and have solved) the problem of "copying the link." I have a solution that works for me roughly 95% of the time. I haven't shared that solution with you yet because I respect the fact that you are working on another solution with another person. If that path doesn't prove fruitful, you can ask me for my solution instead. I respect the path that you are currently going down, so I'm not posting my solution yet.

You haven't shown me your macro yet for how you got Vivaldi to copy the link, so I'm not sure how you did it and I'd appreciate learning how you did it.

13

I haven't seen any follow-up on this -- is there a way to to what I'm after using KM or otherwise?

14

Sorry, I somehow missed your comment earlier about be willing to accept my imperfect (95%) solution.

I just upgraded my Mac to an M3 so I've been busy with that. That's my excuse. Since I haven't transferred my KM macros to my new Mac yet, I'll try to rewrite my short solution here.

As usual, when a solution proves very difficult to find, I often resort to sneaky uses of Find Image in KM to solve things. Any time you use Find Image, there are reliability issues. That's why I said my method would be only 95% reliable. The exact reliability will depend on certain fine-tuning like screen resolution, application zoom level, fuzziness specified, and so on. So when I give you my solution, it will require some fine-tuning by you to reach 95%. You might even reach 100%. I can help you with the fine tuning.

When I wrote my solution, I thought you wanted a macro to copy the link into a variable. Now that I've reread your question, it seems that you don't really want that. All you want to do is click on the link. So that reduces the size of my solution to a single action. If you want the more sophisticated solution, let me know. I can make this macro more powerful.

Remember, you said you want your macro to be used by a single hotkey "when you find the hyperlink." In other words, you will search for the hyperlink, and when you find it, you want a macro to jump to the link using a hotkey.

I can't give you (i.e., I can't upload) my macro because it contains a Find Image action, and any bitmap for this action has to be copied directly from your screen into your action. The macro probably won't work unless you create it with your own screen capture.

Each app (eg, Safari, Mozilla) may have different color schemes for its search results, moreover, the selected colours can depend upon which website you are using (!!!) and that could mean some planning is required by you. (Even Light Mode and Dark Mode will affect what you need to do here.) I don't have Firefox so I can't even check what colours it uses.

The action is: (and you can trigger it with any hotkey of your choice)

image

Notice in the image above, I have captured some grey and some yellow. This represents the upper left corner of the found text. You might ask, "where is the text in this image?" The answer is that there is none because the text doesn't reach that far. If you look carefully at any paragraph, there is some white space between each line of text. There's just enough white space for this trick to work. Your browser's Zoom level may increase the reliability of this macro because it may allow us to increase the height of the search image.

I think it may be possible to have the macro itself search for the link without human assistance. But you didn't say that you wanted that, so I won't provide that solution.

The number 8 in the action above is something I do to make sure the mouse is pointing over the link. You may not need it, but I'm putting it in there for your safety.

Lastly, you should locate the app called Digital Color Meter on your Mac, and learn how to use it, because that app is extremely useful when trying to capture very small rectangular regions for the Find Image action.

15

Thank you for your time on this, @airy. If you have time for a follow up question: the steps as I understood them :

  1. to select text in firefox,
  2. to use the digital color meter utility to then get the highlighted color
  3. to designate that color in KM 's find image action somehow

Is that right? Maybe to simplify I can create a screenshot of just a small section of color field (between highlighted words for instance) and then use that "image" in the find image action?

16

Those 3 steps look correct, but remember that the Digital Color Meter (DCM) is only there to guide your mouse selection. It's not mandatory. I usually don't use "DCM" any more because I have lots of experience selecting with the mouse right down to the pixel.

Your final paragraph is virtually guaranteed to fail. You cannot do it that way. You must do it my way. The main point to my way is to get two colours with a 90 degree angle inside the captured image. Your way will just return a solid colour and will end up scoring a huge number of false positives. If you want to deal with false positives, go ahead. But my way will usually work correctly.

Drag the image capture directly into the KM Find Image action. Do not use any image editor in this process.

17

Okay, understood and thank you.