New version breaks click at found image?

Questions & Suggestions
1

I just upgraded to KM version 9 (always happy to support my favorite utility), but it seems that some unexpected behavior has seeped in regarding click at found image.

I use KM to help me schedule my tweets in Buffer and click into various fields and automatically do what I would otherwise have to do manually. Specifically, it clicks at the found image corresponding to "What would you like to share?" on the screen. Normally this works fine, but in KM9, something is weird. It's hard to know what's happening for sure, but as the script is supposed to click into my image, the mouse cursor ends up near the top of the screen instead, just under the Filemaker Pro menu (I am also running Filemaker). The erroneous location of the mouse causes Filemaker to come to the front instead of Chrome, breaking everything that should be happening.

Is anyone else experiencing a similar issue? Is there anything I can try to help debug the issue?

Showing two screenshots, one my on screen box that I am trying to click on (the "What would you like to share?" area), and also my Click on found image script command that has broken.

17
Screen Shot 2019-08-18 at 0.28.17.jpg1480Ă—756 54.7 KB

48
Screen Shot 2019-08-18 at 0.35.48.jpg842Ă—556 64.5 KB

2

I heard someone say in another post that in KM9 it might help to recapture your images for some of your actions. I can't confirm that's your solution, I'm still thinking about it. I will do my best to help you. I'm pretty handy with Find Image.

3

My first comment, as a Find Image expert, is don't "target" the text (especially light grey text). You've got this nice big blue Tweet icon to the left of the place you want to click. Find Image always works better with solid graphics like that. So in my opinion your action should be clicking "50 pixels to the right" of the Tweet icon.

I doubt that that's your problem, and I'm still thinking, but that's a potential hazard you can fix.

1 Like
4

A good way to debug this is to duplicate your action (the one you displayed above) so that you have two in a row, then put a PAUSE 1 second action between them, and change the first one to MOVE instead of CLICK. That will tell you if the problem is that action or not. I don't think it will be the problem, but this is a good method to find out if it is.

5

I'm not aware of any bugs in KM9 "click at Found Image", although there have been numerous changes related to Found Image, so it is definitely possible.

May I suggest that your use "Found Image" as a solution of last resort. While it is very powerful, it is also somewhat fragile, and numerous things/changes can cause it to fail.

If this is a native macOS app, then here's what I'd suggest, in order of preference:

  1. If the app has a menu item that will do what you want, then use the KM Select or Show a Menu Item action.
  2. If the app has a shortcut key that will do what you want, use the KM Type a Keystroke action.
  3. If the app is scriptable (has a Scripting Dictionary), then use AppleScript or JXA to do what you want.
  4. Use AppleScript UI scripting to access (get/set) the specific window elements of interest)
  5. Last resort, then use KM Found Image , but follow these guidelines:
    1. Choose the largest image with the most contrast in the window of interest.
    2. Whenever possible, set the image search area to be "Frontmost Window", or even better, a specific small area
    3. Usually it is best to move/click the mouse relative the the "found image's center", not "corner"

If you're still having problems, then come on back with what you have tried, and any follow-up questions you have.

6

I had this same problem with KM9, it broke all my found image shortcuts, sending the cursor off to the top left of the screen.
However I found you can fix it.
There's a drop down menu to the right of the 'fuzziness' slider, if you set that to 'Best' rather than 'Unique', which is the default, the KM8 created shorcuts will work again.
I'm not sure what has changed in KM9 in relation to that parameter as I don't have it on any of my machines any more?
Its a bit tedious going through and changing every instance in every shortcut as I have loads, but its better than having to re-create them all.

7

That's a great catch, if correct.

8

All mine are working here. :wink:
I wonder what the change is between Unique and Best?
The others I can understand like leftmost etc.

9

Unique requires that there be only and exactly one match on the specified area/screen. Best doesn't require exactly one. As a result Unique is more likely to fail. I'm not sure if that explains your "top of screen" issue. But you asked what the difference was.

10

All Mouse Click at Found Image actions were Unique in v8, so I am confused as to the solution requiring a change to “Best”.

What I suspect is the fuzz level has changed on some of the matches, resulting in a match that used to be unique now now being. I would like to understand this change better, so if there are any such actions that changed behaviour, and that use images that would be available with default applications, I would love to see the actions that fail now that work in v8.

I also wonder wether an alternate solution to switching to “Best” is to adjust the fuzz level slightly. The Display option should show the required fuzz levels for the various matches.

11

Yes, I learned that, losing some hours but finally figuring it out. Came here to tell everyone but you've already got it!

12

For me, changing them to Unique (which they were all set to) to Best made things magically work. So problem solved!

13

The fuzz level was one of the first things I tried to get them working again and that did not fix the problem. however all is fine with the change to 'Best'.