FIND IMAGE wish list... 7 thoughts

Questions & Suggestions
1

My favourite action (and its closely related “condition”) is FIND IMAGE. I use it a lot, so I probably have a longer wish list for it than others may have.

  1. The fuzziness slidebar value is a constant, not a variable, which limits what my program can do with it. I would like to be able to have my program adjust fuzziness values until it can find the object. (I did read the post about changing this into a variable, but I can still put it on my wish list here.)
  2. The bitmap to be searched for is a constant, not a variable, which severely limits what I can do with it. I end up writing lots of duplicate macros because I can’t put a variable into this “Image Well.” Duplicate macro management wastes a lot of my time. One time I dragged an object into the image well and I saw something that looked more like a variable or file name in the Image Well (!), but I think that was a glitch; I can’t repeat it.
  3. It lets me search the screen but won’t let me search an image stored in a clipboard, which limits it a lot. If I process an image with IMAGEMAGICK, how to I get it back into KM so I can search for something in an image?
  4. The FOREACH FOUNDIMAGE is very helpful, but it only lets me search TOP DOWN or LEFT RIGHT, when sometimes I need to do BOTTOM to TOP or RIGHT to LEFT. Also, I’d like this flag to be a variable rather than a constant, so that I don’t have to write macros to handle these 4 or 5 cases.
  5. The FIND IMAGE action returns a value if the result is unique, or returns a single (best?) result if I uncheck “must be unique” but I would really prefer it returned all the images, one per line of result text. Perhaps the best result should be returned in the first line, but since it can also sort from TOP or from LEFT, that should be possible here too.
  6. When I use the AREA component, I have to pass four separate values instead of a single variable which could contain all the data it needs. KM has some good support for rectangle handling, but it’s not found here.
  7. It’s not obvious what the “Display” option box is for, and the help page for this action doesn’t contain the word “display” so I can’t figure it out. It isn’t obvious to me.

It’s a VERY efficient action/condition. I would call it magical. I love it. I use it so much it could be the reason I am experiencing memory leaks with the KM engine.

If I could pick a single item that’s most important to me, it would be #2, to allow a filename to be inserted into the Image Well. That would effectively make it a variable, without polluting the variable environment, and with minimal effect on the UI. But #5 would be very helpful and looks fairly easy to implement.

4 Likes
2

“Display” is covered by this:

“Optionally highlight Found Matches
This can be helpful in determining the right amount of fuzz to use.
It can also show when an image is being found in more than one location.”

Ie - when Display is checked, when the macro runs, the found image on screen is highlighted in green, so you can visually see the image that was mapped.

3

Oh! I thought “display” was a noun and not a verb. As in “LED Display”. So I tried it, and it’s fascinating. I may have tried it before but didn’t notice the effect so I ignored it since.

Even for someone like me who adores this action, and has read that page many times, it wasn’t obvious what that description was talking about there. Sure I’m an idiot, but I’m probably not the only idiot in the world who needs to read this in the documentation: “the Display option box in this action causes the KM engine to briefly place a green highlight box around the found image. It’s very handy to assist in debugging.”

1 Like
4

Hey @Sleepy,

See this:

Find Image on Screen using Clipboard Image?

-Chris

5

As I indicated in bullet #1 I had read that post (which covers both #1 and #2). Peter N Lewis starts off by saying no, then goes on to give an XML example. I didn’t think his example would work because he starts off by saying it’s not possible. Now that you got me to relook at that post, it does look like it might be possible to solve the first 3 problems I mention. I really had to study that example to understand it. So thanks for forcing me to relook at it. Still, it would be nice if my first 6 bullets could be solved by a modification to the action. I still don’t understand why replacing a slider with 100 positions with a numeric value with 100 values is a hard thing to do.

6

Hey @Sleepy,

No, he doesn't. He says:

He says you can't do it with normal Keyboard Maestro actions, BUT it might be possible via an alternate method.

He then goes on to show how how to use that alternate method in a subsequent post.

People don't intentionally post complex macros demonstrating how to do something that doesn't work.

If you haven't designed and built software with Xcode and Objective-C/Swift then you have no basis for understanding.

Even a relatively trivial change is minutes (or hours) of work and has the potential to break things that will cause many more hours of work.

Peter is very happy to entertain suggestions, but he has his own notions of how to do things.

If you really want a feature you need to make a strong case for it and do more than just say “I want feature x...” The latter rarely works.

That said I personally don't like slider controls very much – especially when they don't have an alternate numerical input field.

And that said – you may or may not have noted that you can use a variable input for the fuzzy attribute when building the action from XML.

-Chris

2 Likes
7

Yikes, you seem to be taking this very personally, like I’m attacking you, so I won’t attempt to respond to any of your points. However I will answer your last sentence, by saying that yes I did notice that the post showed how to use a variable for a fuzz factor, and I indicated this when I said “it does look like it might be possible to solve the first 3 problems I mention”.

I wasn’t attacking you, I even thanked you.

8

@Sleepy, I can assure that Chris (@ccstone) is NOT taking it personally, nor is he attacking you. Chris is one of the most helpful and knowledgeable members of the KM community.

He is trying to offer you some suggestions that would make you more likely of being successful in using KM, and in requesting KM features.

BTW, you need not respond to my post.

Good luck with Keyboard Maestro. It is very powerful, and has many, many features, that often take even an avid user of KM some time to fully discover.

3 Likes
9

The Find Image facility is already a very complex action to configure, and often is part of similarly very complex actions such as the Mouse Click action.

This makes it very hard to add further complexity to the UI in the form os things like:

  • Allow the image to come from an alternate source (clipboard, file, etc)
  • Allow the fuzz to come from an expression
  • Search images (from clipboard, file, etc) instead of the screen (actually, this one is more possible since there is already a popup menu for the source).

While there are obviously cases for these, it's hard to add more complexity.

Perhaps an alternative solution to this is to expose the Find Image facility via AppleScript (rather than by creating XML based actions as is currently required).

This covers items 1 and 2 and somewhat 3.

4 is possible, I'll not it does - it primarily comes to a cost/benefit in terms of how much time is saves how many people vs how much time it takes to implement (not much in this case, but this is the first request I've heard for this so perhaps not much on either side of the equation).

Item 5 is best done yourself using the Find Image collection, since you can control the order of the results there too. But I'll ponder whether I can include options for the results - it would have to then list all sorts of options to make sense: must be unique, best, left most, top most, all best first, all left first, etc. A lot of complexity for me to implement for something you can do with the Find Image collection relatively easily.

Item 6 is handled most easily by using the .left, .right, .width, .height modifiers on a rectangle variable. It's unlikely I'd add another option to allow it to be specified with a single variable instead of four calculations with the same variable.

I believe the wiki documentation has been enhanced somewhat to better describe the Display option.

I agree, Item 2 would be the most useful of the additions, but as I say, the complexity of adding some sort of UI to toggle between selecting a file and an image well sounds painful to me.

10

I appreciate your thoughtful reply. I realize that only a small fraction of my ideas ever see implementation, but that won't stop me from suggesting things.

1 Like
11

Often even when ideas are not specifically implemented, they lead to other facilities that resolve the underlying issue, so they are always worth suggesting.

1 Like
12

Reviving this topic.

In the If Then Else action, would it be possible to add a new dropdown list to the Found Image Condition "in area" option that allows us to specify which corner of the front window is the search area rectangle's X,Y coords relative to?

The condition would read:

If the screen contains image ==[image]== in area ==[x, y, width, height]== relative to the front window's ==[selected corner]==.

You can already do this via some calculations, but I think this would simplify things.

13

I think that's a valid idea. I would like it, too. But I also admit it introduces some complications, because what the user sees as the front window is not always the same thing as what macOS considers to be the front window.

14

KM already provides this for the mouse actions. You can specify a set of coordinates relative to a given corner of the front window or the screen.

15

That is true. But the poster is asking for this to be included in the IF condition for Find Image. It's missing there. Do you see a use for it there?

16

I am the original poster so... yes :wink:

17

Oops. Sorry! Sometimes I miss that sort of thing.

1 Like
18

You can already do this using the Area selection, in combination with the WINDOW function.

Eg:

WINDOW(1,Top)
WINDOW(1,Right)-30
50
30