When it finds too many matches that are all as good as each other it gives up.
Text all looks the same to an image matching algorithm. You have walls of text all over the screen which all looks more or less the same.
I have said previously that you should restrict the area of the search and you will get better results, but the search is still over all screens.
Your options are:
Restrict the area that is being searched and Keyboard Maestro will be able to use more CPU intensive searching over the smaller area and thus use more accurate searching.
Search for something that is not just plain text
From an image matching point of view, this is an approximation of what your screen looks like:
This is 100% incorrect. It is NOT FINDING ANY IMAGES AT ALL.
a) If it was finding multiple options, I would see green squares everywhere.
b) If your thesis was correct the macro would never run, whereas it runs perfectly most of the time and fails after somewhere between 10 and 50 iterations, with no fundamental deviations in the amount of text that is available on the screen for sampling. sorry but you just dont get it.
To illustrate this, I have made you another video, with the amount of text available for sampling increased by maybe 400%. If your thesis was correct, the find image step would catastrophically fail. It does not fail at all, it finds the word with a unique match of 4% and zero other matches found. Please review: https://ipb.life/IMG_3347.mov
Mm, if Peter's image is supposed to be an analogy for what KM would see if it were a human, then nothing would ever be found. It's probably meant differently
But it also shows that colored icons are much easier to see.
Edit: Have you tried whether the same error occurs (or not) when searching for images with other software, for example BTT?
No. My thesis was, if you reread it at the top of this thread, that the search would be "unreliable." I did not say it would "catastrophically fail." Don't change my words. The fact that you proved that it worked once does not disprove my thesis that it would be "unreliable." I definitely said "unreliable" and that's what I meant.
But nevertheless I like your latest video. Thanks. So I tried to replicate it to the best that I could. Sure enough, the text I searched for was "found." I even ran the macro 100 times, and like your video, there was no failure. I even moved the window around, running it 100 more times, and like your video, there was no failure. So how are we supposed to fix something that we can't duplicate? Maybe it has something to do with your specific hardware. As I suggested above, it might have to do with the Intel Macs that you are using. More on that, below.
That's not correct. If the library that the KM Engine uses was finding multiple options, it would return no data to the KM Engine. I think I've explained this several times. Let me try to explain it one more time in the next paragraph.
The code that does all this "image searching" is probably not part of the source code for the KM Engine. It's probably a public domain library that is included in the KM Engine. And that library has to work with several kinds of video cards on both Intel and M1 architectures. This is why I asked you much earlier about your video card and whether you are using Intel Macs. And to my surprise, I was actually right. All your Macs are Intel. This is currently the main suspect. I have an old Intel Mac, (which does not have the same video card as yours) but today I was too lazy to set it up to try to replicate your code. (Actually, it wasn't so much laziness as sick coming back from the hospital. Nothing too serious.)
The fact that all your machines "have different video cards" does not "rule out" (your words) the idea that the problem may be caused by a video card (or, importantly, the library that KM uses to fetch images from those video cards.) I may never be able to replicate your problem because I won't likely have any of your video cards.
As @peternlewis explains, bits of text simply don't lend themselves to image recognition – no image defined by character edges has a practically useful chance of turning out to be unique within a large textual swathe of such edges.
You only need image detection for (maybe) that last bit. For locating the file in the first place you would be much better off simply searching for it, and opening it, getting the text, and pasting into Telegram can be done with KM actions.
Once you've a file path it is much easier to open the container in a new, smaller, Finder window which would give you a much-reduced area to search should you still need image detection for eg a drag'n'drop. But do you need to? I don't use Telegram, but this page clearly says that the desktop client can do uploads via a standard "Open" dialog -- and that's easily driven using the path to the file.
Yes, that would require rewriting chunks of your macros. But it would result in much better macros. Computers are really good at searching and processing text, but they are nowhere near as good as us at doing the same with images (there's a reason Captcha gets you to do that to prove you're human...).
If you do want to carry on trying to get image detection to work, upload a small macro that will demonstrate the problem -- you do need to upload the macro itself, not just images, so people are working for the same baseline. I can find a Catalina machine to test on, others might be able to as well.
We'll also need to know what version of KM is on the Catalina machine.
In the meantime you should carry on but with Activity Monitor open. There's no evidence so far that it is actually KM that's "freezing" -- it could be that you've another occasional process that suddenly hogs your CPU and blocks KM. It might be that, because of your old (and now out of Apple support) OSs there's a problem with the WindowServer process or similar and KM can't get an updated "view" of your screens until you force if with your "jiggle". And there's a bunch more things it might be...
That you're having this problem when using KM doesn't necessarily mean it's KM causing it -- KM's behaviour may only be the symptom.
Many thanks.
The trouble with the path to the file approach is that there are over 10,000 content folders in the social media CMS, how can I program over 10,000 file paths into one macro? The other option would be to upload all the file paths to a database and have KM read from the database. But even so, that's a fair chunk of work and requires a new layer of complexity; every time there is a change to the folders I would need to update the database. Alternatively, I would be back at the same point of drag and drop with the folders in order to create a file path on the fly.
This is why I said that it is not about just remaking a macro, which would be no real challenge. The issue is a massive database of content in order to make that content database play nice with text based file finding. Hope that makes sense.
I agree about the cause and the symptom, but had eliminated other possible causes on the basis that the issue persists across different OS's on different hardware with different monitors. KM is 11.0.2 on all machines.
there is "evidence" that it is freezing. I uploaded that in the video. But I agree that it is not proof of cause.
That's interesting about the activity monitor. But again, with different OS's on different machines, I dont know what the process might be that is giving identical behavior on both setups. It's possible but the commonality here is KM...
You're not currently programming 10,000 folder locations for image search in you macro, right? Your'e doing this: "The macro goes through the folders one at a time, using a down arrow keystroke..."
Keyboard Maestro can do that much more effectively. For Each can step through folders, or files in a folder, or both.
Without actually seeing you macro, though, it's very hard to give more detailed advice, but from what you've described so far, there's nothing that says you'd have to create a 10,000 item folder database, nor use found images.
But again, it's hard to be any more helpful without a better understanding of the current process. I understand that the macro is proprietary, but if you were to describe the workflow, perhaps someone could offer up solutions that would be more reliable—and probably much faster—than the method you're trying to use now.
If you don't want such help, that's your call, of course. But if you continue to use image recognition on the old Intel Macs running old macOS versions, I think you are going to continue to have problems with the image recognition phase of the macro.
Airy: I did not say that you said it would catastrophically fail. I was not in fact responding to you, I was responding to peternlewis.
I then demonstrated clearly that the image finder is capable of working perfectly well and that the amount of text on the screen is not the issue, whereas if it WAS the issue, it would be demonstrable as being such. I ran a test for the thesis and proved it incorrect. That is all.
It does not go through all 10,000 folders in one use. I have over 10,000 folders in total. There's no simple way to set a single work flow that addresses all of the folders that any given macro would need to. It would be an utter nightmare to set up, would probably never work correctly and doesn't even deal with what would happen when something gets moved. It is a fluid environment.
It's not that I don't want help, its just that the proposed solution is not viable in this instance.
What I would like to do is figure out why KM is failing at the image finding task and fix it. That and only that. If someone can help with that, then I'm all ears.
Is there a known issue with Intel Macs or is there not? I've been told that it is not a known issue several times. If the issue was actually with Intel Macs, I rather think that someone else of the countless KM users over the years would have flushed it out by now. I'm not the only person who has ever used KM on an intel mac since the image finder feature was introduced, am I?
Assuming they're all under one parent hard drive, and share something in common in naming, KM could find them all for you without trouble. But again, you don't seem interested in a solution that involves moving away from found image, so that's fine.
What people have said, including the actual author of Keyboard Maestro, is that finding text onscreen is troublesome for any number of reasons. You're also using old CPUs with an old version of the OS, which introduces further unknowns.
As a test, if you have access to one, I'd suggest trying your macros on an Apple silicon Mac running Sonoma or Sequoia. See how the image find works there, as compared to the Intel Macs.
At this point, I'm out of ideas, so I'm signing off this thread barring any new information.
So would we. But the likelihood, given the lack of reports of this happening for others, is that this is specific to your setups and/or what you are doing -- so it will be very difficult for anyone else to troubleshoot.
Again -- create a minimum viable demonstration macro and upload it. Then other people can try and reproduce the problem.
You don't. How does your current macro find the file? By looking for a "visual" match with a file name. How much easier is it for the Finder to do a text match? All you have to do is work out the appropriate search. From your earlier description you seem to be looking in every sub-folder of a parent, and that's really easy to do either directly in KM with a "For Each" action or a Shell Script action using mdfind. You could even base your search on the "target" of the current Finder window.
Plenty of examples of these methods have been posted on this Forum, but if you need more help we'll need some specifics.
There's all sorts of OS processes running all the time, many of which have hardly changed since OS X was introduced so will be the same across your machines -- in fact the biggest chance of your machines being different is if you had an old PowerPC G5, an Intel, and a third on Apple Silicon. So there's way more commonality than you think, and certainly more than "they all run KM".
I have to point out that column view is a performance killer, and using that may be the real problem. The Finder is constantly checking every item visible in a window to see if it has been updated, and in column view that's every item in every column. It's a known problem when connecting to a remote volume and, given your huge number of directories plus older hardware and software, could be the issue here. The best solution would be to change your workflow so you didn't need column view, but if you've a spare machine and some time you could try your macro on a subset of your data to see if that improves things.
