Based on my personal experience, that could actually be a factor.
I can't really think of anything specific to your case, but I use find image a lot, and I've found that I can get it to work 99% of the time if I do some combination of:
use Pause until...the screen contains image (because this gives me more fine-grained control over how long it'll wait before "giving up"). It also helps with troubleshooting because I can watch it in real time to see if it can find something.
if I want to click on something, I'll pass the %FoundImage% to the click action
trying to look in the smallest space possible gives me the best results. e.g. instead of searching in all screens or even main screen, I'll try to search within a window or even a smaller space. I know this isn't always practical, but the smaller the search area, the better/faster it works
once in a while, even if I took the screenshot for the found image on the same computer/monitor/resolution, taking a new screenshot helps. And/or I'll take multiple screenshots when/if the original one fails, and add it to the search (e.g. instead of looking for just the original or the new screenshot, look for either ("both").
related to #4, if you're looking for text, I wonder if font smoothing can be a factor, in which case multiple screenshots might help.
Searching for text is always going to be problematic because text is displayed using sub-pixel anti-aliasing so the actual pixels change as the text moves around the screen, and because text is inherently all similar in appearance to all other text (this is why blurring text is generally safe to use for security purposes because text is so similar in appearance. Further, the modern system has a tendency to change the background of windows based on the things behind them which will degrade the match (and might result in other areas on the screen being a better match than the one you expect). Turning off those facilities (such as enabling Reduce Transparency) can help get consistent image matching.
To ensure reliable image matching, try to avoid small gray blobs that look like everything else on the screen. Also, minimize the search area as Keyboard Maestro uses extra speed optimizations which reduce accuracy when it is searching the entire screen, and removes them when the search area is smaller.
If possible, rather than search just for text, search for something near by that has some color and thus more uniqueness.
Also, not the exact error you get when the search fails, and leave Display on if a specific image match is failing as that will give you more information to work with as to what the exact issue is.
I wish you could have seen the video, so that you would have noticed that it started working the moment he covered up a window that contained a lot of text. As soon as the screen was "less busy," it suddenly found lots of matches.
I can still view the video. So maybe it's a video codec problem in your browser.
That's all very interesting. Note that it finds the image many times in a row in a loop, reporting it as 1%-10% accurate, to then suddenly not finding any matches at all. Which seems unlikely just from aliasing, although I do agree that this is a possible source of error. But when it jams it is not finding any other possible matches either. This is completely unlike the scenario where it finds 20 green squares all over the screen with varying percentages. In this case, it completely freezes and finds no matches at all. Although it seems to be locking up and storing all the matches, so that when I move the window all the clicks are suddenly "released"
This is very practical. I have sometimes tried just creating a new screenshot, and that often fixes the situation 'for a while'.
I had not thought of a) 'pause until' or b) creating more than 1 screenshot and creating an 'either' scenario - will give those a go! Great ideas.
I'm leaning towards something to do with font smoothing. It's the most rational explanation for why all it needs is for the window to be moved by a few px for the macro to work again. Oddly it is noticeably better behaved if the top bar of the window is not touching the top of the screen.
yes it works perfectly during testing. then in a loop situation where it is going through a set of say 100 folders and finding files with certain characters in the file names, it will break at some point.
To illustrate on the example in the video of how I use the image finder:
I have a file naming system for my social media assets. I have over 10,000 social media posts and each lives in its own folder. Telegram does not have an automated scheduling tool, so I created my own with KM. It goes into one folder at a time (using a down arrow keystroke), locates the image and drags it into the app, then opens the txt file that is marked TELEGRAM.txt , copies the caption, pastes it in, then sets the date and time and moves on.
It will do maybe 10 or 20 posts and then fail to find the file, despite the file name containing the identical string.
So yes for this particular use case I could possibly figure out OCR, though there are many items in the folders and I really don't fancy going through thousands of folders locating all the various other things that might break the OCR.
The image only occurs in one place. It's not throwing me multiple options, its freezing up.
Otherwise I would see green squares everywhere.
I often hide other apps so as to avoid picking the wrong option.
But the main thing is that it works a number of times in a row and then fails without any changes to anything aside from viewing a different subfolder with the same number of files in it.
The problem persists at all levels of fuzziness.
The emoji idea sounds worth a shot also. I have tens of thousands of files that would need updating, I am NOT doing that manually, but might be able to batch some of it with Name Mangler.
It started working when I moved the window. The issue persists when the windows are on different monitors and things are not getting covered/uncovered; I just superimposed them on one so as to be able to make a video for you.
Because this is KM and you can control almost everything, and you say the problem is often when the window is near screen top:
Have your macro hide all apps but Finder. Close all windows but one. Open the window to the folder you want to use, and set its size and location as 1200x800 at screen center (or whatever you want). Now your find image can work on a defined portion of the screen, and it will always be the same.
With that said, does Telegram not support enclosing an image via some sort of Insert menu item? If so, that would be a much easier way to handle this: You wouldn't need drag. And from what you described, you wouldn't need image recognition.
We'd need a more full description of what you're doing, but this seems like something that would be easily handled without drags or finding images, just using KM's ability to work with the file system (assuming Telegram supports a file-based insert).
But even if it doesn't, control the variables you can, and put the Finder window at the same spot in the same size every time before you try image finding.
That statement suggests to me that you have misunderstood my diagnosis, (regarding "too many matches") so I'll retire from this topic now. I did my best.
You said "without any changes" then you said your screen views a different subfolder, completely changing your screen. This is the problem. When the screen changes, the number of matches changes, resulting in "too many matches".
That's a change. You previously said "various levels" and now you are saying "all levels." I can only work with the words that you give me.
Again, I can only work with what you give me. Based on what you've given me so far, I stand by my conclusions.
I'm sorry but you have misunderstood. There is only one match when the macro is working, and none when it isn't. This of course depends where I set the fuzziness, but literally wherever I set it from 1% to 99% and all points in between, the error persists just the same.
Issue persists with all apps hidden except finder.
I have made another video. Really trying to show you with as much clarity as humanly possible what is going on. https://ipb.life/IMG_3345.MOV
The video works in Firefox.
Telegram image scheduling typically works via drag and drop. There is a way to attach a file but not doing it - that would entail great complexity and an entirely new system of labeling to enable the finder function to work on 10,000+ folders. No thanks. There's nothing inherently arcane about my system that needs to be changed, and there is 10+ years work that I don't really want to have to edit.
The drag and drop part of the macro is working flawlessly every time. Aside from the app's intermittent breakdown in functionality and failure to locate ANY images, even in screens and windows where the images were previously found with 1%.
This is the point nobody seems to understand. It's not that it is not finding the right image. It is not finding ANY IMAGE. Move the window by a few pixels and it finds it again. Without any changes being made to any of the elements of the window, folder, anything. I'm not changing anything.
I also use drag and drop for many, many other operations. This was just one example.
Anyway hope what I am saying makes sense!
@ROOK Because I'm a dummy who can't build complicated macros, I often search for images.
As someone already said, this picture is found much better
than this picture
This works best
You search on all screens. Is that absolutely necessary? Have you tried what happens if you only search in the focused window?
And another somewhat absurd suggestion: You say that the image is always found if you move the window slightly before searching. Then do that. I mean, of course, let KM do it. This slight shifting of the front window should be easy to incorporate into your macro.
Hi, thanks but we already went over this, testing on just a window and the error persists. It's all in the thread.
Also it is not that it requires the window to move before searching. It works fine until it breaks and then it unbreaks if you move the window. Not the same thing.
I will try adding the emojis. Thanks for the suggestion. But I don't think this is specifically an issue with finding, because it finds it with a report of a 1% match, and then when it breaks, it fails to find any images at all. No matter the fuzziness setting. So it isnt specifically that image that cannot be found. It is any image.
I feel like I am just saying the same things over and over. Does anyone from the company come on these boards? I just want to file a bug report because it is so very clearly a problem with the app. I don't want a band aid fix that takes weeks of work to set up, because that is so very far from ideal with 10+ years of work that would need modification in order to use a workaround. It's a nightmare. I jus want a paid app that does what it claims it can do. And that shouldn't be something that means I have some kind of problem, that's just a normal and reasonable expectation surely. I've made a video. The video surely shows erroneous behavior from the app. And that's what's up.
The author of Keyboard Maestro has already replied in this thread, with excellent advice.
Hence my advice to get rid of the Found Image step if what you are looking for is text. But if you're stuck on using found images, you will have to modify something in your setup to make those words more visible to Found Image.
It is completely incomprehensible why it should be necessary to move the window ...
But if this is the case: the window must be moved for it to work again. Then it would be fairly logical to move the window before each search (with KM) so that it (always) works, no?
This is of course a strange workaround, but if it were that important to me, I would try that.
There is no "general" bug with image detection -- many of us are using it successfully many times a day (an hour, in some cases).
There are, however, plenty of case-specific gotchas because of how the OS renders the screen -- and it looks like you're falling foul of one here. Unfortunately your second(?) video's failures are happening out of shot. The first video is -- weird. Almost like KM pauses detection -- or the OS pauses screen rendering -- until you move the window (notice how that, after you've moved it, even the title bar of the KM Editor gets highlighted). What OS is that particular device running?
Does the problem persist if you always have a 10-pixel gap between the top of the window and the menu bar? Or do you break the cycle by moving the window down and, later, it gets stuck again and you can break it by moving the window up?
Why do you need image detection? Is it simply setting the initial target for a drag'n'drop action? Or is it also reacting to the arrival of that file (i.e. you're constantly scanning, waiting for the file to appear)? I'm thinking you might be able to select the file, then use the selection highlight as a more obvious target for the image detection -- a minimal change to your current workflow.
Yes it pauses detection. It gets brain freeze. This is happening in both High Sierra and Catalina on various machines with different monitor types. I have multiple computers running simultaneously because KM can only run one macro on one computer at a time.
It's not specifically that the window needs to be down. It just needs to be messed around with until it unfreezes KM's broken brain, and generally speaking, the macro seems more likely to fail when the windows are right at the top of the screen.
I appreciate the insight into the fact that workflows are not simple to "just do differently". It's like telling Tesla to "just use a different factory" if one of their robots isn't working right. Philosophizing over whether this is an XY problem is nice, but the proposed solution is impractical except as an absolute last resort.
I do need to find images. For many different macros, as mentioned. For just one of the examples - I have over 10,000 folders with social media posts. It locates the elements within the folders, which always contain different numbers of items, and then performs various operations regarding the content management system. Scheduling on various social media sites, uploading posts to wordpress, interacting with apps including photoshop and LLMs.
I have evolved the folder system for the content management over the last 10 years and the folders are also in Dropbox, meaning that manual actions can be performed by anyone on the team, as is often necessary. All of the file names are unique but contain tags, for example the txt file containing the caption for the Telegram post is marked with the suffix - TELEGRAM.txt, as in the example macro I demonstrated. The macro goes through the folders one at a time, using a down arrow keystroke, locates the file, opens it, copy pastes the text, switches to Telegram, pastes it into the caption field, then goes back to the folder, drags and drops the image, then schedules the post.
If all else fails here I might be able to use Name Mangler to add an emoji to the file suffixes, which is a short term fix, but ultimately is a band aid solution.
