Find Image on Screen Action not working in Keyboard Maestro (v10.0.1)

Find Image on Screen Action not working in Keyboard Maestro (v10.0.1)

I just noticed that a macro of mine, which uses the 'Find image on Screen' is not working any more. I created several new screenshoots of the image to find, without any luck.

Attached you find a macro, which searches for the run button in Keyboard Maestro and on my machine it always says 'Not found.'

Does have anyone an idea what's going on?

Thanks!

Test Image.kmmacros (15 KB)

Is your monitor standard resolution or 4K? I have found that when capturing screen images from a 4K screen, there are different ways, and some ways end up providing the wrong DPI on the image. You didn't mention which method you used to capture your image.

Does the image you captured have contrasting colours? For example, when I try search for white squares on an image of a chessboard, it doesn't work very well. Contrast is important. If you are willing to show the image, that may help.

Try increasing the fuzz to maximum and see if that helps. I also recommend turning on the Display Flag when troubleshooting.

Try switching to a completely different high contrast image to see if that works, eg, a black corner on a white background.

Try rebooting the KM engine. When was the last time you did that? I find that when I do that sometimes the image search works much better. Believe it or not.

These are my best ideas based on the information you provided.

Hey Sleepy,

thanks for your detailed response. I've got a 2020 13" MacBook Pro with Retina Display.

The image itself is just black text on white background. I used Keyboard Maestro to drag my signature beneath the found image. It worked for a long time without a problem. But then it stopped working. Today I restarted the Keyboard Maestro Engine several times. Without luck. I also revoke and granted the screen recording permissions in the settings.

I played around with the fuzziness and with that I was able to catch the test image. So something else must be wrong.

Here you find a sample image of what I am searching for with Keyboard Maestro:

Bildschirmfoto 2021-11-27 um 15.06.15

You answered most of my questions, correctly, but you didn't say how you copied the text as an image and moved that image into the Find Image action. What method did you use to do this? Since you use a high res monitor, this is important because the wrong method can create an image of the wrong size creating mismatches for complex images like yours.

Oh sorry, I used the macOS built-in screenshot feature to take an image of the text in PDFpenPro then I opened the image in Preview and copied it in the clipboard. And pasted it then into the Keyboard Maestro action.

When I used a similar approach to you (using preview) I got a conversion that caused the same kind of error. I recommend that you use the macOS feature to take the snapshot and drag it directly into the Find Image action rather than using Preview. Then tell me if that fixes it. If I'm right, Preview introduces a DPI conversion issue.

3 Likes

Yes, awesome! Thank you Sleepy. That was it. Now it finds the image without a problem.

I hope the world now recognizes me as a KM demigod. Glad it worked. I kinda understand the problem, but it's mostly from years of experience.

The lesson to learn is do NOT use Preview. Drag images directly when using high res monitors. (Although if the image is just a high contrast border, Preview's DPI manipulation won't hurt.)

Basically, when you used Preview, it DOUBLED the size of your image, which is why the match failed.

2 Likes

Good catch but don't push it Sleepy! :slightly_smiling_face:

1 Like

It's just my sense of humour in action. No real pride problem here.

Indeed my hero of the day! Thanks again.

I know man. I get it as my humor is very similar. :sunglasses:

I use Find Image so much, I don't need to read the wiki for information about it, but I think this issue about DPI and Preview should be placed into the wiki because I'm sure it will cause many people grief. I spent days troubleshooting this issue when I first discovered it.

https://wiki.keyboardmaestro.com/action/Find_Image_on_Screen

This isn't the only KM action where image scaling can cause problems. To help illustrate the problem, look at this image from System Preferences:

Notice that the fourth of five mini-boxes is checked with a blue box. It's because of this setting that macOS scales images. I think if users pick the fifth box, there would be no problem with scaling. But most users probably don't like that resolution. But I didn't mention this to @pulli because I didn't think he would want to change this setting.

I just noticed a tag called "bug" on this thread. Technically, this isn't a KM bug. It's an issue of understanding and dealing with macOS screen scaling.