Found Images Suddenly Broken in Google Chrome

I updated to the official build number you quote above. I had previously installed the beta, which didn’t fix my problems so I uninstalled and reinstalled previous version (public release). Today I simply used Chrome’s check for updates feature to get the new newest release; again, it matches the number you quote above. No joy.

For the first part of my macro, if I allow the image to be searched for on all screens, it is unique enough to be found, although it takes up to 10 seconds depending on how much I have displayed on my six screens; the next part of the macro is an if statement looking for multiple conditions; and it depends heavily on looking at only the Chrome (Hangouts) window in the frontmost position, or matching the window title. The image it is targeting in any part of the if statement is very tiny and tends to match a considerable number of objects open in other windows on additional screens, so I get too many matches and the macro fails; if I tell it to look only in the frontmost window, or a window with a particular title, the chrome bug rears its ugly head and fails to provide it to KM and KM pukes.

It is becoming tiresome.

Thanks for your interest.

Frederico,

Which version of MacOS? I'm running Keyboard Maestro 8.2.4 on macOS 10.12.6

Same. I am starting to wonder if it has something to do with my six displays; I’ve just had a horrendous week and I’m exhausted; but I intend to adjust the macros to see how it responds with objects on the main screen. I also realized I have not logged out and log in; I suppose that’s worth trying.

Maybe even rebooting.

I am seeing the same issue.
See

Running Google Chrome 69.0.3497.100 (3497.100) on macOS 10.12.6

I've also confirmed since the issue has returned again, even with the latest release of Chrome and KM, that simply changing the macro action from "front window" to "main screen" enables successful results and operation.

I’ve been extremely busy this past week, and quite honestly, that breaking of my daily workflow has had me avoiding my Mac as a result; I take and make a tremendous number of daily calls via Hangouts via Chrome, and to not do so with a single key stroke Is horrible when I am forced to mouse click mouse click mouse click mouse click mouse click mouse click just to take or make a call.

Since I have six screens, I have to figure out how to try to deploy location of image by screen index instead; the thing is it’s not always consistent, and since chrome is not returning information correctly, I got stonewalled on my to to 3 BRIEFR times at working around it.

Thanks for confirmation that it will work if I can add additional if statement trees to get there.

Hey Frederico,

Front window is broken presently in the Keyboard Maestro action.

BUT -- I was able to substitute the window frame of the front window in Chrome instead.

Google -- Find Image in Front Window Tests.kmmacros (27 KB)

I tested with click found image as well, and it works.

I'm testing with the newest Chrome Beta on macOS 10.12.6.

Google Chrome Beta -- v70.0.3538.22 (Official Build) beta (64-bit)

I only have 1 screen, so I can't be sure how it will work with 6.

I think as long as you're working in the front window you'll be okay, but you'll have to test to be sure.

-Chris

I was able to change several parts of my macro from window ID to screen ID, and some of them now work properly; but other parts still refuse to work. It's also very slow compared to when it worked by defining the window ID by name or number. One of the worst parts to be broken is the ability to answer an incoming call with a single keystroke. I got the bit working so I can make an outgoing call, but most of my calls lately have been inbound and it's driving me insane; it takes about six mouse clicks to actually answer a call, and my Parkinson's makes mousing around, especially when you're counting rings, very agitating. I hope the Chrome team gets this figured out soon.

okay, your area trick is working great; I just have to install it in about eight places.

@peternlewis I'm not convinced this is all on Chrome; if KM can find and manipulate and move a Chrome window based on either its title (Hangouts) or position (front), and then fail only on Click On Found Image (Front, Hangouts), but can Click On Found Image by [area, screen ID, main screen, all screens], then why are we blaming Chrome? KM clearly knows Chrome Windows by title or position or index (always returns true) for every action other than Click on Image (by title…); wouldn't that suggest it is KM failing inside that particular action?

Capturing a window is not the same as capturing the area of the window. The obvious difference is the shadow.

I had a closer look and basically the issue is that the Accessibility name for the window does not appear to match the CG window name. For example, for www.stairways.com, the accessibility name is:

Stairways Software: Stairways Software: Excellence In Software – Only with a Mac - Google Chrome

but the CG window name is:

Stairways Software: Stairways Software: Excellence In Software – Only with a Mac

so basically the “ - Google Chrome” is only included in the accessibility name. I don't even really know how they manage this since it should be the same.

Since there is no direct way to find the window except by matching the name and position and size of the accessibility window information to the CG window list, having different names results means the window cannot be found.

Why or how they have managed to get the CG window information and accessibility window information out of sync I have no idea.

I think I understand your response; but I just want to clarify that KM can identify the same window(s) to successfully move, resize, minimize, etc., based on position (front) or title; so it still seems very odd to me that KM fails to identify that same window in the same macro one action later if finding an image within said window is the action.

And I, out of habit with automation quirks, rarely use exact title parameters, if it's not absolutely necessary; I always use 'contains' or 'begins with'; so, in my case, the window I'm after always begins or at least contains 'Google Hangouts', but indeed changes based on whatever account might be logged in; thus my macro is portable to other users.

KM first looks for that window, and brings it to front; if it's not there, it uses a subroutine/macro to open; it then moves it and sizes it to a specific location, then it either conditionally begins or takes a phone call, based on image conditions. So KM has no problem finding the window by name and position in the first place, but borks on images using identical identifiers. I'm not exactly certain how the CG/Accessibility explanation fits this scenario, given you are using title as example, but I'll trust you that it's Google's fault, since it worked prior to Chrome 69.xxx; hopefully whatever they broke will get fixed, else I hope other users getting nailed by this and spending far too much time troubleshooting the completely baffling problem and useless error messages (given the above stated success with other actions) find this thread.

I'm super grateful for @ccstone 's solution/workaround to the problem, which is a feature of that action I've never explored or needed before. I try to avoid image-based actions whenever possible, even if more work is required, but that may ultimately be both faster and more reliable due to changes in artwork.

Love this forum; love the heroes in it. Cheers. -- F

Because the system has two different window lists accessible to Keyboard Maestro - one via Accessibility, and the window manager window list which is the one that allows window image capture. The accessibility one is fine, but the window manager list has a different title. And there is no correspondence between the two lists (no way to go from an accessibility window to the window manager window list entry). So Keyboard Maestro searches based on matching size and title. But somehow Chrome has managed a different title in the window manager list. Hence there is no matching window.

I trust that is clear?

Sure, this is very sensible, but it works only because you know that there will only be one matching window that contains the substring. Keyboard Maestro doesn't know what part of the window title is important or what sort of match is “good enough” - and it doesn't normally have to because the window titles should always match exactly. Honestly I don't even know how Google Chrome managed to get them different.

I think so, kinda; you're telling me that even though both title classes should be the same, one is clearly different; apparently in my case, the phrase 'Google Hangouts' isn't appearing in the CG list; and for Found Image, you cannot use only position/ID? That's where I'm getting lost, because position (front) and index ID are both a singular option in that action. I'm just not entirely grokking why, if you have position, you must also have title (partial or exact) and size (I kinda can guess on size since you need to know what area to scan).

Ultimately, it doesn't matter; it's broken and I've reported what little I could to Google; I hope you and others who actually know what is really going on have given them better information.

I apologize if I'm seemingly very dense; I'm in a lot of pain this week and not everything is getting through.

Further, and I should have explained more, but it didn't seem relevant, I have if statements in front of the Hangouts macros that account for the idea that more than one window/tab with that phrase in the title at the beginning or otherwise might exist. It's not possible for the Hangouts Chrome app to have more than one window open, so that much is never a concern, and any other windows that match partial title can quickly be checked for a unique (missing) element and dismissed until it finds the correct target.

Thanks

This should be worked around for the next version of Keyboard Maestro (or resolved anytime the lunatics who caused Google Chrome to add “ - Google Chrome” to the end of every window’s accessibility title revert their ridiculous change).

2 Likes

What version is it fixed in? I am still seeing it with KM 8.2.4 and Chrome 74.0.3729.157.

It's not fixed just yet; it'll be in version 9.0, which will be released whenever Peter deems it ready (hopefully sometime soon :crossed_fingers:)

The “next” version.

Version releases can be tracked at:

https://www.stairways.com/press/

Anything after 6 August 2018 referring to “next version” will be 9.0, when it is released.

And presumably Chrome has not fixed the issue yet either, and probably will not.

Thanks, Peter! I wouldn't have thought to look there for KM's release history, so that's good to have as a reference. As for Chrome, I can confirm that Google has indeed not fixed the issue yet, and I agree that at this point, it's probably safe to assume that they may never do so.

1 Like

Just wanted to chime in that this fix still works. I had working macros that searched the "front window" for an image (front window being Google Chrome) and it was working perfectly fine for months. All of a sudden, stopped working. Verified that it was detecting the correct window as the "front window", about went crazy trying to figure this out.

Then, set the "frontWindow" variable to %WindowFrame%1%, and searched for the image in "area" related to frontWindow[1], [2], [3], and [4] and works like a charm. Didn't read enough into the thread to see where the fault lies, but just wanted to share that this is still a viable solution.

Thanks!