The following action worked in macOS 15.1, but in 15.1.1 it doesn't work. See below log file for exact error. I've examined my permissions for the App and Engine and all seemed to be enabled. I've even performed the following:
tccutil reset Accessibility
as recommend in the User guide. IMHO I think it's a Safari Security issue, but what do I know.
Thanks in advance...
Action called
Move Mouse
At (0,0) from the center of the found image in front window.
Image from File “~/Library/Mobile Documents/com~apple~CloudDocs/Key…” (Best). Fuzz: 75%
Log File
2024-11-24 07:12:49 Execute macro “Download Field Guide copy” from trigger Duplicate Macro Palette
2024-11-24 07:12:49 Action 820376 failed: Move or Click Mouse no unique image found
2024-11-24 07:12:49 Move or Click Mouse no unique image found. Macro “Download Field Guide copy” cancelled (while executing Move Mouse to (0,0) from the Center of the Found Image).
2024-11-24 07:19:24 Execute macro “Download Field Guide copy” from trigger Duplicate Macro Palette
2024-11-24 07:19:24 HScreenArea::CaptureCGImage failed to capture front window
2024-11-24 07:19:24 MatchImage::Match failed to create source image: failed to capture front window
2024-11-24 07:19:24 Action 820376 failed: Click at Found Image failed with error failed to capture front window
2024-11-24 07:19:24 Click at Found Image failed with error failed to capture front window. Macro “Download Field Guide copy” cancelled (while executing Move Mouse to (0,0) from the Center of the Found Image).
One of your error messages says "failed to capture front window" but another message says "no unique image found." Well, the first thing you have to do is tell us which application (and version of it) was your front window when you executed this action. Some applications, I believe, don't allow you to capture the front window. I think I read that some versions of Chrome have this problem, for example.
But if that's not the problem, I have a lot of other ideas that each have a 10% chance of fixing your problem. Since these ideas take only a few minutes to test, it's probably a good idea to try them out. (Not just in this case, but any time Find Image is failing.)
In the log file it says "no unique image found." The key word here might be "unique". It could be something as simple as you having more than one copy of the image on the screen at the time of the action's execution. (Or zero copies of the image.) There's no way to test that by looking at the log file. One thing you can do is turn on the "Display" toggle in the action so that all copies of the image are highlighted when the action runs. If you see exactly one green rectangle, then this theory is incorrect. So I recommend that you try this.
Another possibility is that you have changed the screen's resolution during the OS patch. Or that you changed the setting called "Larger Text / More Space" in the display pane of the System Settings. You may want to ask yourself if these settings have changed.
Another possibility is that you need to restart the KM Engine. Although Find Image is wonderful and amazing, I have found cases where restarting the engine fixed the problem (typically when the image being found is fairly monochromatic. So try restating the engine.
Another possibility is that the action is somehow broken and to fix it you need to delete the entire action and re-create it using KM's editor.
If you try all these things, I would estimate a 50% chance that one of them will fix your problem.
Are you talking about the Press a Button action in Interface Control?
It's a graphic that has a very very long URL associated with it.
I've tried all the steps listed. The common denominator is that KM can't grab the image of the Safari window. The image is unique and should match since I created the image file from a screen grab of the item I want to click on. This used to work perfectly.
Image matching can be finicky. Have you tried taking a new screenshot of the image directly to the clipboard (⇧⌃⌘4) and pasting it into your macro? Sometimes I have to finesse the settings, changing it from unique to best for instance and dragging the slider to adjust the sensitivity.
If that means you tried all the steps that I listed, (I'm not sure if that's what you meant) then it would be helpful to me if you were more explicit. For example, did you try enabling the Display toggle and did it show exactly one green rectangle on the screen? This is crucial information. I need to know exactly how many green rectangles appeared on your screen.
I switched to Firefox and things worked as expected.
I tried everything you suggested. The problem seems to be that Safari prevents KM from having access to the screen or its pixels. Display never had the chance to work, since it failed before it could. The old script works by switching to Firefox, just like it used to do in Safari.
Uhh, I don't have the knowledge or time to dig through the Javascript. I had to be so pragmatic, but I gotta get my job done.
You wouldn't have to dig through it, but if you posted the HTML, it'd be pretty easy to see if there were a unique ID we could use to click. It'd be a one-action one-line thing to add, and it'd get rid of image matching.
But if you have it working, that's the important part!
You've got two different errors in the log -- one for "no unique image found", another for "failed to capture front window". So it looks like Safari is sometimes preventing screen capture (second case), but when it does work the image can't be found (first).
I've got the latest Safari happily detecting images on web pages, so neither problem is a general one. Perhaps something specific to that site? If it's a public site, post the URL so others can try.
This used to work perfectly.