Find Image - Not Finding Anything Useful (Big Sur)

Questions & Suggestions
, ,
#1

Hi all,

I'm having trouble getting Find Image to work. Both Find Image on Screen and Click at Found Image just won't return anything useful.

I am trying to detect the Mic icon in MS Teams to show the on/off status on a Stream Deck, and clicking the icon when pressing the Stream Deck button. The Stream Deck part is working fine, just cannot find images on screen.

I have tried using Command-Ctrl-Shift-4 and pasting into the image well, screen capture to PNG files (no JPG), making sure the image size is correct...

When I use other images I get some hits. Like when copying the left part of Finder I get one hit, but the mouse (0, 0) position is way off. Obviously I can correct with -300, but this was just to try if anything works at all.
For smaller images the fuzzyness needs to be way up (80-90%) and then it finds multiple occurrences which visibly do not seem to be anything like the image in the inage well. And the (0, 0) is also way off.

I am using Big Sur, which in the promo promises to glas the title bar.

Anyone with suggestions how to improve the Image Finding?

Can't get 'find the image' work :(
#2

Hey @Frans,

All the transparency and bleed-through has to be a problem.

I'm not using Big Sur, so I can't test myself...

-Chris

#3

Have you tried switching from Unique to Best?

#4

Hi, tanks for the suggestions.

@ccstone, switched to reduced transparency (System Preferences -> Accessibility -> Display -> Reduce transparency) but did not make a difference. Maybe other options you know of?

@thoffman666 I have tried switching. With fuzzy way up and big images it does find images (multiple), showing in orange with Unique indicating failure, and showing in green with Best, Top most etc., but these findings aren't visibly correct, just seem random on the screen.
It also only finds images in screens, not in any of the window settings (the front window or window with title etc.).

I am using a MacBook retina and an external 4K screen over usb-c. Both have different resolutions.

Any help is appreciated. Maybe anyone using image find on Big Sur successfully?

#5

Well, I’m using Big Sur on both my Macbook Pro and my Retina iMac, and my images are being found. It sounds like you’ve tried all the reasonable tweaks. I’m not sure at this point.

#6

@thoffman666 thanks for the confirmation, so it should not be Big Sur. It is stated as compatible. I am running a Trial version stil but it should not have limitations.

I'll play around with some permissions, toggling off and on again. If that doesn't work I'll give their support a try.

#7

I found the issue and solution.

Issue: The wiki mentions Accessibility Permission Problem, and correctly allowing access in System Preferences -> Security & Privacy preferences -> Accessibility and enable both Keyboard Maestro and Keyboard Maestro Engine.
When playing around with disabling and re-enabling these, suddenly a popup appeared asking Screen recording rights (sorry, translating from Dutch so might be slightly different naming). Turns out another required checkbox was not set, not allowing KM access to the screen?!?. Keyboard Maestro was not complaining about not having the proper access before I messed with the checkboxes. Also the wiki does not mention the screen recording permission requirement. Maybe Apple changed something?

Solution: In System Preferences -> Security & Privacy preferences go to Screen recording and enable access for Keyboard Maestro there as well.
Now it finds the images just fine with fuzzyness around 20%.

3 Likes
Found Image Not Working, Alternative Solution or Fix?
KM9 Can't Find the Image
#8

@peternlewis please note.

#9

The system is meant to prompt you to enable it.

However the system is buggy, and so if you ask for window images, it will fail, but if you ask for the entire screen image it will prompt you.

This has been reported to Apple. Given their desire to improve quality across the board I am sure it will be fixed soon (and since sarcasm does not carry well in written language, I will note that yes, this is entirely sarcastic, but who knows, I've been surprised by Apple fixing things before, it might happen).

3 Likes
#10

Thanks Frans! That solution worked for me!

1 Like
#11

Thanks alot!

2 Likes
#12

@peternlewis how can we trigger the prompt to add the permission? I believe we need to add both KM and KM engine to the screen recording, but when I tried to add we can only select KM from application folder. How can I add KM engine?

#13

The Keyboard Maestro Engine is within the Keyboard Maestro.app, you can get to it using the Finder, control click and Open Package.

But Screen Recording permissions does not let you manually add an application, so you have to try other things, like:

  • Screen Capture the desktop
  • Screen Capture a window
  • Restart
  • Remove and delete the Keyboard Maestro.app and download a fresh copy.
#14

I'm running KM 8.2.4 on macOS Big Sur 11.6.4. I've walked through the tips in this topic all the way to the end (post from @peternlewis Oct 1, 2021) and have managed to get permissions tweaked to the extent that Screen Capture Main Screen works in a macro (I used that command to get a prompt to add Keyboard Maestro Engine.app under Screen Recording - thanks especially for that tip!). But I still see no indication that Find Image on Screen is even attempting to do anything - there's no message about a failed search, and the variable I attempted to set comes up empty.

This is an old problem - in the past (a few years ago) I used this function and then one day, it just stopped working - presumably following an OS update, but I'm not sure. I'm wondering if this issue would be solved by updating to the current version of KM? Or should I also be able to get it to work with my older version?

#15

Use the Screen Capture action and capture the front window, and an area of the screen, and verify that they are working. This is the best way to verify that Keyboard Maestro has appropriate permissions to see stuff on the screen. The window should be any non-Apple, non-Keyboard Maestro window, and the area on the screen should include that window.

Assuming that works, then you know Keyboard Maestro can see the screen. Then use Screen Capture to capture an area of the screen with some unique, colorful stuff, like the Reply bytton on this page. Then paste that in to a Find Image on Screen action, and enable the Display option, and use a reasonable large fuzz setting, and click Try and see if it detects the image.

1 Like
#16

@peternlewis Thanks for the quick reply! I finally just now had a chance to try your suggestions. They worked pretty effortlessly. I'm not sure what I was doing differently before - now it seems I can't get it to not work. So thanks - very much appreciated!

1 Like
#17

This worked for me too! Thank you!