Find Image Doesn't Work

Tinkerer · March 4, 2023, 5:29pm

I cannot get Find Images to work. I have looked at multiple relevant topics in this forum and none of the suggestions have worked. Every time I run my macro, I get the "Macro Cancelled" message.

MacOS version: 10.15.7
KM version: 10.2
Browsers tried (I am manipulating a browser page): Chrome 110.0.5481.177, Safari 15.0 (15612.1.29.41.4, 15612)

Relevant parts of macro:

Relevant system permissions:

I am not clear if I need both the ""Find Image"" macro and the ""Move on Click from Center of Image"" macro. Regardless, it doesn't work if I have one macro or both in my script.

I have tried this dozens of times, tweaking things like the fuzziness level, etc., and it fails 100% of the time. As part of this, I also tried using the image well. That didn't matter. I've also tried looking in the front window vs. the main screen and other options, and none of them work. When I execute the script, I am always doing it from the relevant window in my browser.

This is what I see in the Engine log:

2023-03-04 12:11:10 Execute macro “Template for new Medium article” from trigger The Hot Key F3 is pressed
2023-03-04 12:11:14 Action 2367 failed: Move or Click Mouse no unique image found
2023-03-04 12:11:14 Move or Click Mouse no unique image found. Macro “Template for new Medium article” cancelled (while executing Move and Click at (0,0) from the Center of the Found Image).

It would be very helpful if there were debugging level log messages that said why Action 2367 failed.

Any help appreciated. I would really like to get this to work.

peternlewis · March 5, 2023, 3:41am

No, you only need the Click action.

The first thing to do is to ensure the system is not lying and that Keyboard Maestro actually does have Screen Recording permission. Use a Screen Capture action to capture the screen to the clipboard, including something other than the desktop and Keyboard Maestro windows, and verify that it works correctly.

Then remove the Find Image action, and enable Display on the click action and then Keyboard Maestro will show you where on the screen it is matching.

I would suggest temporarily switching the action to include the image in the action, screen capture a section of the screen you want to match and paste it in. That removes any issues with the file and its contents (things like DPI could mean the file and the screen are not scaled the same way which will stop the action from working).

There is, in the Engine.log file. Select Help ➤ Open Logs Folder for a record of the complete message.

You can also use the editor menu View ➤ Select Last Aborted Action to view the specific failing action.

Tinkerer · March 5, 2023, 5:17pm

Hi Peter,

Thanks for your help with this.

I removed the Find Image macro.
I tested a Screen Capture action. Captured to system clipboard then pasted it into a document. That worked.
I changed the script to paste the image I captured in Step 2 to my document.
I changed the ""Move on Click from Center of Image"" action to load the image from Step 2 into the image well.
I get the same error whenever I run the macro.

I run IDrive for backups. Could that daemon possibly be incompatible?

Macro:

Image:

Error Log:
2023-03-05 11:20:46 Execute macro “Template for new Medium article” from trigger The Hot Key F3 is pressed
2023-03-05 11:20:48 Action 2367 failed: Move or Click Mouse no unique image found
2023-03-05 11:20:48 Move or Click Mouse no unique image found. Macro “Template for new Medium article” cancelled (while executing Move and Click at (0,0) from the Center of the Found Image).
2023-03-05 11:22:59 Execute macro “Template for new Medium article” from trigger The Hot Key F3 is pressed
2023-03-05 11:23:01 Action 2367 failed: Move or Click Mouse no unique image found
2023-03-05 11:23:01 Move or Click Mouse no unique image found. Macro “Template for new Medium article” cancelled (while executing Move and Click at (0,0) from the Center of the Found Image).

My Notes document before running the macro:

My Notes document after running the macro:

My system log around the time of the second macro execution as shown in the Engine Log.

Mar 5 11:22:56 Pasture com.apple.xpc.launchd[1] (homebrew.mxcl.php): Service only ran for 0 seconds. Pushing respawn out by 10 seconds.
Mar 5 11:23:02 Pasture com.apple.xpc.launchd[1] (com.apple.mdworker.shared.05000000-0700-0000-0000-000000000000[50887]): Service exited due to SIGKILL | sent by mds[144]
Mar 5 11:23:02 Pasture com.apple.xpc.launchd[1] (com.apple.mdworker.shared.0D000000-0200-0000-0000-000000000000[50919]): Service exited due to SIGKILL | sent by mds[144]

Attempted again at 11:59:36
2023-03-05 11:59:36 Execute macro “Template for new Medium article” from trigger The Hot Key F3 is pressed
2023-03-05 11:59:38 Action 2367 failed: Move or Click Mouse no unique image found
2023-03-05 11:59:38 Move or Click Mouse no unique image found. Macro “Template for new Medium article” cancelled

Here is my corresponding Mac error log:

peternlewis · March 6, 2023, 7:27am

No. If Keyboard Maestro can capture the screen, then Keyboard Maestro can find the matching area.

It looks like the image you are searching for is very large, which means it will be very unlikely to match anywhere as small changes on the screen will likely render it not a match. For example, the image you are searching for includes the clock, so that will definitely be changing. Also status menus will be changing, etc. What are you trying to click on?

Try this, adjust your macro to just have a capture of a small area visible in a window, something unique, colorful, about a couple cm square. Eg your avatar icon in the forum.

Copy it using the system screen capture to clipboard, and then paste it in to the image well (don't use the image above).

Then see if you can get the action matching the image.

Another thing to try, instead of the click action, create a macro with the If Then Else action and the Found Image condition, make sure it is set to Display, and make sure Keyboard Maestro is evaluation conditions.

Then the editor will repeatedly check the condition, searching the screen, and highlighting the matches.

Generally, don't bother looking at the Console log, it is so full of drivel these days as to be largely useless. The Keyboard Maestro Engine log will tell you everything you need to know and has significantly less drivel.

Pretty much everything you show from the Console log is unrelated to Keyboard Maestro except the specific lines that also appear in the Engine.log file.

Nige_S · March 6, 2023, 11:42am

It looks like you have a screenshot of your entire desktop -- how many pixels wide and high is that?

It looks like you are then pasting that image into a Notes document -- how many pixels wide and high is it now?

Do those images still match? My guess is no, which is why KM can't find it.

You may have a similar problem with your original macro -- it looks like you are pasting an image into a web page. If that page is re-rendering it to a different size to the original, eg a smaller preview version for display, you'll have trouble getting a match.

Tinkerer · March 6, 2023, 3:12pm

Thanks for your help.

I only used that image when following Peter's suggestions for debugging this issue.

My real image is much smaller, but still fairly large. But, you're right. I wasn't aware that size was relevant to the matching algorithm.

I was able to correct this by using the same image, but capturing a small distinct subset of it for match. And it works. Thanks again for your helop.

Tinkerer · March 6, 2023, 3:17pm

Hi Peter,

Many thanks for your help. I didn't realize that size was relevant to the matching algorithm.

I was able to correct this problem by capturing a small distinct subset of the large image, and using that in the match action.

The other issue I had is my image was not fully visible in the window. I eliminated some stuff above it. That corrected the issue. So I have this now working.

Thanks again for helping me.

Nige_S · March 6, 2023, 4:26pm

It's a literal, pixel-for-pixel (although modified by the "fuzziness" you set) attempt to find a match. Peter's suggestion was simply to verify that screen recording was working, Privacy entitlements were correctly set, and so on -- but I can see how you could have interpreted that the way you did.

There's lots of gotchas in image matching -- transparency, slight colour changes, areas hilighting because they're active or the pointer is over them -- but KM generally does a good job considering.

Tinkerer · March 6, 2023, 4:35pm

Useful info for me. Thanks, I got it working, so on to debugging the next thing that doesn't work. I'm new to KM so at the left edge of the learning curve. It does seem like a powerful tool.

Nige_S · March 6, 2023, 5:17pm

Golden Forum advice is: Keep the manual bookmarked and check it and the accompanying Wiki pages when you get stuck -- but if still stuck after 5 minutes, post your problem here.

And have a read of Tip: How Do I Get The Best Answer in the Shortest Time? and How to Post/Upload your Macros and Scripts. It really does help others help you if they know where you're starting from, what your end goal is, and if they can test your actual macro rather than guess at things from a screenshot of a piece of it.

Tinkerer · March 6, 2023, 10:13pm

Added bookmark to manual wiki and have been using them extensively. Very helpful.

I will read those two links. Many thanks!

I've been able to solve the remaining issues for this macro on my own so it's working. Will probably writ many more though since this tool really saves me time.

peternlewis · March 8, 2023, 2:58am

Also, remember that holding down the option key and selecting from many popups/menus will take you directly to the wiki reference on the subject (eg, hold down the option key and select from the Add Trigger menu, or while selecting an action, or in the Insert Token menus).

Find Image Doesn't Work

Options