Inconsistent Image Recognition with Keyboard Maestro – Seeking Advice

Hey everyone,

I've been a long-time user of Keyboard Maestro, particularly relying on its image recognition functionality for various automations. Generally, I've got it set up correctly, and it works like a charm, identifying objects as expected.

However, I've been encountering a strange and frustrating issue. After I close the app or shut down my computer, upon restarting, the image recognition doesn't trigger at all, even though I've made no changes to the settings. This inconsistency is baffling because I test it extensively and everything works perfectly fine, but then, out of the blue, it just stops responding.

Has anyone else experienced this? It's like the settings hold correctly for a session, and then somehow, they seem to "forget" their configuration, requiring me to retune the whole setup. I'm at a loss here and could really use some insights.

Here's what I've tried so far:

  • Double-checking that the triggers and actions are set up correctly.
  • Testing the image recognition multiple times in a session, where it works without fail.
  • Ensuring there are no updates or changes to my system that could affect Keyboard Maestro's performance.

But, despite all this, the issue persists randomly after a system restart or app relaunch. Any advice or troubleshooting tips would be greatly appreciated. Has anyone solved a similar problem?

Thanks in advance for your help!

You said when it doesn't work you have to "retune the whole setup." I don't know what that means. Define "retune" for me please. This is important because you claim that retuning it fixes it. So if you explain what you are doing that fixes it, I may be able to spot the problem. Better yet, if you are willing, show the Find Image action or condition that you are using.

Thanks for highlighting the need for more details in my previous message.

By 'retuning the whole setup,' I'm referring to adjusting the 'fuzziness' value in the image recognition tool. Typically, I find a 'sweet spot' for this value, where the tool recognizes images with 100% accuracy. However, the issue arises when I restart my computer after a day or two. Even without altering the fuzziness setting, the tool fails to recognize images as it did before. This inconsistency requires me to recalibrate the fuzziness value each time to find a new optimal setting.

I hope this clarifies my earlier statement about the inconsistency in our image recognition system's performance.

is it clear now?

When did this start?

At the moment, newly created found image actions are hardly working for me at all, no matter what fuzziness settings I use, or the search area.

However I have one older found image macro that works consistently.

I can't say if this started with Sonoma or KM11, as I don't use Found Image very often

Okay. Since you haven't indicated which version of KM you are using, I have to ask. Which version of KM are you using? When I was using KM v10, I would sometimes have to restart the engine, and for reasons unknown, Find Image would work much better. Therefore, I recommend that you wait for the problem to occur, restart the KM image, and see if that fixes the problem. If it fixes the problem, then I recommend that you upgrade to v11, where I have not yet experienced that problem.

Your symptom could certainly be caused by using the option "Unique" instead of "Best." Check that setting. It's always harder to troubleshoot any problem when I can't see the code. But I do have a lot of experience with Find Image.

I too have been having problems with recognizing images since November - ever since I upgraded to Sonoma (currently 14.2.1) and with it Keyboard Maestro from ver 10 to 11 (currently 11.0.2). Previously created images are all working fine - it's the ones I have been trying to create since the upgrades. Sometimes, by sliding toward the right it recognizes and highlights (in green, with relatively high numbers in the top left corner like "52%", "80%") an area with the same background, but it's NEVER on the image I placed in the act. And it's not tied to any particular image - it's with ALL of them. Restarts have no affect.

Use the Screen Capture action and verify the screen is being captured.

The most common problems with matching issues are:

  • The search image is the wrong DPI and so the match is made with the search image scaled to a different area than what you expect.
  • The screen image is affected by things that change its appearance (font smoothing, background changes, animations, “vibrancy affects”, etc).

The matching code in Keyboard Maestro is unchanged, so issues related to matching are caused by changes in the display or changes in the image DPIs.

1 Like

Thank you
Okay, so the DPI might have changed during the system upgrade (I had considered this), but the images that preexisted before the system upgrade all are recognized and work fine, so I ruled that out. Again, it's the newly created ones that, while in the same resolution the image is lifted, is not recognized

I kinda remember that the DPI value I had before the system upgrade wasn't an exact option in the new system, so I settled on something close (which happened to be the one marked "default"). If I try to change the DPI now, won't all the preexisting images then fail? I want your thoughts before I go messing things up more.

Ahh. Then you may be being bitten by an issue happening when you do the screen capture that the DPI is not set correctly (it should be 144 for retina images, but can be erroneously 72dpi).

If this is the case, then it should be relatively obvious when the Display option is on because the matched rectangles will be twice the size as the captured area (twice in size in both directions, four times the area).

1 Like

THAT WAS IT!!
I used SET IMAGE DPI action to change the failed images and they all seemed to now work.

Now I can't quite figure out how to get the screenshots grabbed at 144dpi automatically.

1 Like

If you figure that out, I'd love to hear it. It is possible it is third party software doing something (I heard from one person who had Hazel automatically optimise screenshots and that was an issue). Or it could be a bug in macOS (which should be reported via Feedback Assistant).

Possibly it's a Sonoma issue? Mac screenshot resolution on macbook pro changed from 144 to 72 dpi after upgrade to Sonoma - either via keyboard shortcut or terminal command - Ask Different

And possibly, EasyRes may be involved? https://discussions.apple.com/thread/252364143?answerId=254467739022&sortBy=best#254467739022

1 Like

If you use the built-in screenshot tool and capture to clipboard (Shift-Cmd-Ctrl-4 for region capture), then paste into KM, it will be at 144dpi: At least, it is here using Sonoma on my retina laptop.

-rob.

1 Like

BINGO! I love a "workaround" that's actually more efficient than the process I was performing. I never knew about the "CTRL+" version of CMD+SHFT+4 - it eliminates having to dig up the SCREENSHOTS folder (user-defined) to drag the image from, WHILE fixing the DPI issue.

Thank you Peter,
Thank you edjusted,
Thank you Rob

1 Like

Glad to hear it's working for you!

-rob.

I made a group called 'Capture' and added a macro for each of the different screen capture shortcuts. I set that group to display in the menu bar. When I need a specific screen capture, they are all available in a nice drop-down list.