Is there a KM equivalent to the Better Touch Tool action "search for text on screen and move mouse"?

I find the action very useful in Better Touch Tool , especially in cases where found image is unreliable, and have not been able to emulate it in KM.
thank you very much.

I have no idea what BTT can do, but two years ago I wrote a very powerful KM macro that searches for a word on the screen and moves the mouse there.

It's a powerful macro, but it can be difficult to learn to use. You should read that thread and see if it meets your needs.

1 Like

I will look. thanks very much

Hi @ronald,

could you please give me a hint how to find this action in BTT? I even searched the entire manual using different terms but didn't find anything. Or is it a beta feature?

Thanks in advance.

I think I found it mentioned here as being on a BTT alpha (not a beta)...

Either macOS is providing that information (the location) back to BTT, or BTT is using the trickery in my macro above to determine the location. If macOS provides the location information, I would beg The Architect to let us get that value.

Hello @Hans-Peter_Henkel and @Airy

ScreenShot #1
on the left, the name of the action and on the right the configuration parameters. All you have to do afterwards is follow up with a mouse click if necessary.

ScreenShot #2 and #3
I ran the action on this screen search for @Hans-Peter_Henkel and the cursor moved on your name (unfortunately my ScreenShot app does not show the cursor)

@peternlewis - could you add this type of action to your wish list ?- thank you

Mit freundlichen Grüssen

2 Likes

Which version of BTT do you have?

5.444 regular version. I never use alpha.

Thank you so much, @ronald!

This is awesome.

And +1 from me to add this feature to the KM wishlist.

Thanks again. :grinning:

1 Like

If that feature works well, BTT may become the second utility on my Mac. While I enjoy being 100% faithful to KM, this is a feature I really want.

Hmm, the next major release of KM is due soon (I am basing that inference only on historical precedence) so maybe I can hold out until then.

3 Likes

I'm trying to understand this possibly very useful feature: does it actually select the found string?

1 Like

It moves the mouse to the text. You can follow it by a mouse click action. There are many more features, such as specifying which area of the screen to search in case the text in question appears in multiple areas of the screen.

for example, I did a text search for ALYB followed by a mouse click on this forum conversation (last 3 posts)
1- the mouse moved to ALYB
2- the mouse click opened your profile

ScreenShot of this conversation with your profile open

Thank you for your explanation. So it's not possible to select multiple adjacent words by clicking?

1 Like

Sorry, I don't know

if I follow the text search by
1- a double click, I select the first word
2- a triple click, the complete sentence which includes the words in the text search, as per the ScreenShot below in which I search for "select multiple adjacent"

1 Like

That sounds interesting. How exactly do you "search for select multiple adjacent"?

If I'm understanding it right, @ronald is searching for the text "select multiple adjacent". The pointer is then positioned in the middle of that found text's rectangle. A double- or triple-click will then be interpreted by the OS/app as "select the word I'm clicking on" or "select the sentence I'm clicking on", respectively.

@ronald -- What happens if the searched-for string is across a line-break, eg:

So it's not possible to select multiple
adjacent words by clicking?

Does it not find the text, or handle it in some other way?

1 Like

That would be impossible usually, because it's using OCR to read the text. For example if it found the text in a JPEG image on screen, how do you select text in a JPEG image?

If you really want to select the text that you find, then the app has to be processing text, and if that's the case, you can probably use the app's "Find Text" feature to find and select text.

1 Like

Hello @Nige_S @Airy @ALYB @Hans-Peter_Henkel

In summary, after the Find/Search Text action

  • double mouse click to select a single word
  • triple mouse click to select the whole line

how to select multiple words

The issue of selecting multiple words is solved by the very smart @xidiot in the Better Touch Tool Forum who made the following suggestion and screenshot today

The solution is simply to
1- Find the first word(s) on the screen using the Find/Search Text action
2- Double mouse click action to SELECT the first word.
3- add a series of ⇧⌥ → to add words to the selection.
Note to @Nige_S ⇧⌥ → will allow you to select across a line break.

The developer of BTT @andreas_hegenberg mentioned today that he is currently working for better ways to select specific parts of text and AI integration.

image

1 Like