How can I use the information from 'Find Image on Screen' and click at the position where the Image was found?
What am I doing wrong here?
Hey There,
There's a Click at Found Image action.
Spend a few minutes searching the forum for "click image".
Examine this page:
https://wiki.keyboardmaestro.com/action/Find_Image_on_Screen
If you don't find what you need then post back. (See appended macro for hints.)
-Chris
Find Image on Screen.kmmacros (17 KB)
1 Like
Why a so demonstrative/didactic example would not appear directly in the wiki:
https://wiki.keyboardmaestro.com/action/Find_Image_on_Screen
-Alain
1 Like
Alain, count yourself lucky to have found as useful a program as KM, that
has such an active and helpful user community. Alpha contributors like
ccstone and JMichaelTX make KM my go-to app dozens if not hundreds of times
a day.
And then there is the incomparable Peter, the most active program developer
I have ever seen. How he is able to keep adding new features to KM and
still find time to send out 6 responses in a row to my inbox is simply
amazing.
Finally, a wiki is a user-supported enterprise. If you see a hole, such as
the one that you have identified in your posting, then by all means, go in
there and fix it.
From one very satisfied fanboy.
Mike Briggs
1 Like
Hey Alain,
Huh? Did you look at the wiki? 
https://wiki.keyboardmaestro.com/action/Find_Image_on_Screen
(Only so many people contribute to the wiki, and they have only so much time.)
-Chris
2 Likes
Thanks Chris for all your contributions I follow.
- I was not quite sure it is in the general policy of the wiki to illustrate by example(s), but now very happy to be
- I don’t have wiki credentials (but don’t ask Peter for, because I am not so fluent in English (a poor French guy with some risk of misinterpretation…)
Cheers,
-Alain
Alain, if you would like to write up anything you'd like to see in the Wiki, you can can send it to me via PM or post a new topic to my attention, and I'll be happy to be your wiki editor.
2 Likes
Interesting double collaborative approche of wiki documentation: thanks @JMichaelTX for this and your so useful dialogues with @ccstone
1 Like
Just so we are clear, you could use the Move and Click relative to the Found Image instead of relative to the absolute position, but what you have would work if you used Variable.x and Variable.y in the Move and Click mouse action.
The Pause is unnecessary in this case (unless the screen is changing and you want the click to happen after a change or some such).
Hi, your macro is very helpful to me, thank you.
Did you not post an update to that macro? Ver 2, (if you will?)
I thought I saw a find.image.on.screen2.kmmacros or it may have been the second time I downloaded it. !!!
thanx
Hey Troy,
No, nothing new.
The little gray-circled number next to the link indicates how many times it has been clicked.
-Chris
@peternlewis, messing with finding/clicking images. Took a lot of trial and error to figure out a basic task. It may be helpful to see what my experience was:
- After using Find Image on Screen, I was again on a wild goose chase because I thought the value of ActionResult was determined like this [with or without quotes around "OK"--I tried both]:
...when in fact I had to use
...insult to injury is that "ActionResult" is the top entry on the list of variables in the dropbox by text fields, so I assumed I was correct in considering it a variable 
- The next wild goose chase was not knowing why this didn't work:
- then I had to figure out that to use a variable to set the mouse location, I had to choose "absolute position"...
...and then recall that when I started typing a non-numeric character, the boxes would get larger and allow me to type in expressions.
And then I had to figure out NOT to use the % signs in those fields:
...needed to be:
...and heaven knows why those are orange 
Addendum: actually, I should have used .midx and .midy instead of .x and .y:
...and I'm guessing the orange before was because "myImage" hadn't successfully been populated with coordinates yet
SO...
Some recommendations to save pain for users (and forum contributors ):
- ideally testing ActionResult as a variable (as in first picture) would work [although I think your intention would be for "OK" not to be in quotes...]
- the "if/then" could have a separate condition like "last action succeeded" (which would handle at least the true/false case of ActionResult...). This let the user know that the success of the last action could be tested, and would avoid such frustrations as having to recall the term "ActionResult"; having to figure out whether it's a variable or text token (and thus how to test for it); having to figure out/recall whether the result is True/False, Yes/No, 1/0, or OK/[other]; having to figure out whether OK should be in quotes
- if Find Image on Screen does find an image and store in a variable, shouldn't the user be able to use "Use Variable to Set the Mouse Position"?
- for "Move and Click mouse", perhaps have another option in "relative to the": "calculated position". Then the large boxes would be shown by default [and optionally there could be small static text showing examples like myImage.midx and myImage.midy
Hope these are helpful!
Best,
Aaron
That is because you created a variable named ActionResult, which happens to be alphabetically early.
I can sort of understand why you might expect that to work, but it doesn't really make sense. You find an image, that gives you a rectangle where the image is located. Then you want to use that to set the mouse location, but a mouse location is a point, not a rectangle. Which point should it use? Top Left? Center?
Correct, it is a numeric field.
It is a complexity in Keyboard Maestro that there are numeric fields and text fields. It would be better if they were more clearly identified, and yet the identification would have to be subtle enough that it was non-intrusive otherwise the editor would look like a Christmas Tree identifying the different fields. Part of that subtle identification is exactly the first part you mentioned "and then recall that when I started typing a non-numeric character, the boxes would get larger and allow me to type in expressions". They are shown as numbers in part because that makes it clear that they are numeric fields. It would be far easier to code to have them show as text fields. Heck, it would be far easier to code to have them all text fields and force every numeric field to use a %Calculate% token. But I doubt anyone would be happy with that either.
They are orange to let you know that what you have entered is a valid expression (unlike the red of "%myImage.x%), but that the expression is currently invalid because the variable myImage does not have numbers in it. Keyboard Maestro is trying to help you, but by making it clear that what you typed is technically ok, but also that it is not currently valid.
And that is exactly why the Use Variable action cannot simply take a rectangle.
Correct.
The ActionResult variable is of your making. Delete it (and remove uses of it) if you don't want it.
Yes, this is on the todo list, although not done yet.
Yes. But it finds a rectangle, so you need to specify where in the rectangle you want to move/click. Probably the error here is having the Use Variable action set the mouse location at all.
Relative to a calculated position is an interesting idea. I am kind of loath to make that action any more complicated than it already is. There would be no need for the fields to be large, since a numeric offset to a calculated region would still make sense (eg 10,10 in from the top left corner of the rectangle of the found image). And, as noted, those small numeric fields are there for exactly the reason of showing you clearly what kind of field they are.
I could add a relative to a variable, but it's hard to imagine how I would word that so that it would be clear. It's possible I could make some pseudo entries in there, say "variable’s top right corner" and have it switch to "absolute position" and fill the fields in with Variable.MaxX, Variable.y.
Thanks for the feedback.
Hi Peter,
Thanks for the explanations and thoughts! I wondered whether “ActionResult” as a variable was my doing :).
You could keep “use variable to set mouse position,” and the action could have a dropdown: upper left, center, etc. [I know a pain to make the interface though :)]
For “Move and Click”'s little numeric boxes that expand to fields for calculations: I appreciate you made the boxes responsive to what the user enters, but I think users generally will assume they can only put in numbers because the boxes are so small, and KM’s interference is so excellent that it wouldn’t provide tiny little boxes if the user were to be able to input something that wouldn’t fit in it :). So my vote is still to add “calculated position” (as another item in “relative to the”) [or at least some kind of text indicating that variables & calculations are supported as well!]
Thanks as always for the great work!
Argh! I think I just realized that "Find Image" doesn't set %ActionResult% !
It looks like instead of testing %ActionResult%, I need to test for whether the variable holding the result is empty! This seems to work:
Correct. In this case, the action does not fail per se, it simply does not find the image and reports that with an empty result. It could fail, maybe even it should fail, but it pre-dates being able to adjust the behaviour of a failing action, so actions that have a way of reporting failure typically don’t fail as such.
Makes sense–thanks!
Wow. This is really useful to know.
They are orange to let you know that what you have entered is a valid expression (unlike the red of "%myImage.x%), but that the expression is currently invalid because the variable myImage does not have numbers in it. Keyboard Maestro is trying to help you, but by making it clear that what you typed is technically ok, but also that it is not currently valid.
Also, I just spent quite a while trying to debug why, when I first ran Find Image and then ran Click at Found Image, I got nothing. I completely missed that Click at Found Image had a place to put an image.
Has anyone compiled a list of the "100 Most Common Newbie Mistakes in KBM"?
Nope, but you might be in a good position to do that.
I can start you off:
- Asking a question before searching the forum to see if it’s already been answered.
(Not trying to be sarky, btw)
3 Likes
- Asking a question before searching the forum to see if it’s already been answered.
Yep. Common problem in Help forums.
The biggest background issue that feeds this problem is as a newbie thinking you don't know enough of the answer to ask the "right" question. Of course nearly everyone was in that position initially.
When I searched the Wiki for "how to click on a found image" I got a bunch of stuff that was so full of formatting or whatever that I didn't see that the answer was buried in there. There is no Wiki page for the action "Click at Found Image".
I actually had more luck trying to write a question about it and having the forum software ask me if a couple of topics were what I was asking about. It got me to this thread.
2 Likes