Click at Found Image Action Detects Button but Fails to Actually Click It

I found my way to this current thread from the linked thread below, referenced above.

I have been wrestling for days with a simple problem of finding a Chrome Extension Icon on my Chrome toolbar and clicking it. The macro has worked for days and now has stopped working. I think it was because I tried to use the "frontmost window" in Chrome because the macro was sometimes finding the icon in the wrong window. (I don't remember, I don't keep notes of every change, especially not ones that seem "obviously right".) I have tried everything I could think of, including putting a pause in between moving to the button and clicking on it. But I couldn't figure out what the issue was.

What I found on that other thread, the one above about "Simulate Pause Until Any of Multiple Images Are Found...", was the notation that

As of Ver 75.0.3770.142, Google Chrome has had a bug in it for ~1 yr+ that causes the KM criteria for "FrontMost Window" to fail.

THAT seems to be my problem. I had been presuming -- for accumulated HOURS of debugging -- that the Frontmost Window was the natural way of identifying where I wanted to find the image of the button icon. And yes, in the macro, the Go button would indeed go there, and yes, when I ran the macro with Display on, I could see the green highlight saying "1%" for less than 1% fuzz needed to make a match. Everything has been looking right. And it has not been working.

Knowing that there is a glitch where KBM has trouble locating the frontmost window in Chrome let me change it to "screen of frontmost window" and that works -- AS LONG AS I don't have any other Chrome windows in that screen where that icon might be visible. If that's the case, then I get orange highlights on the buttons indicating that more than one image has been found. I want the image in the frontmost window, but I can't set that, it seems, because of the bug, so we go around the circle again.

What seems to be working for now is to use "frontmost window's screen"; it seems Chrome will allow KBM to know where the frontmost window is, even if it won't share the size, or whatever the bug actually is.

For the future

What I would really love is for bugs like this to not be buried as a comment deep inside an answer to a problem that I may not have. I would love for the Wiki page on the actions that are affected to have links to a post that describes the bug. Then if it's ever fixed, just that post needs to be update even if removing the links gets overlooked.

For example, in the thread above, JM comments that KBM has a "design flaw" in the logic around setting the %FoundImage% token. There is a link to his bug report where Peter explains the design logic, JM argues his point again, Peter explains more deeply, and then Peter announces a fix that looks like it it will solve JM's issue, which JM confirms, at least for his test case.

The fact that JM linked his "design flaw" comment to the bug report is what made it possible to figure out that the problem explored in this thread has been (mostly?) solved. Thanks for that thoroughness, JM.

What I'm asking for is the same kind attention to detail whenever bugs are discovered -- in this case the bug (or "design flaw") in Chrome that limits the usefulness of the "Frontmost Window" option. For something like this, that severely impacts KBM's ability to function as designed and described, it would be very useful to have a reference in the Wiki, wherever "Frontmost Window" is defined, to the bug report that tracks the current state of this conflict with Chrome.

I'm remembering the words of a VP of Engineering that I worked for a couple of decades ago: "If it's not accurately documented, you've wasted your time building it."

1 Like

I’ve found locate by image too finicky and stick with absolute positioning. What makes this work is to have window positioning macros to easily and reliably position windows. This is an important part of my daily actions.

Putting a window positioning action for when an app activates in the global group gets things going from the start and having a shortcut or palette on screen to relocate if needed or put in a positioning action step within a longer sequence has served me well.

1 Like