The Front Application changes

Questions & Suggestions
1

Is it possible to add more description to the Wiki on this pause action. I kept thinking it was just looking for some window or significant changes in the front application but it is really saying "Pause until the front application changes to another application" if there is something I am missing or really it can look for window or visual changes that happen in the front application that might be something I am looking for but that doesn't seem to be what it does.

https://wiki.keyboardmaestro.com/action/Pause_Until_Change

2

I think you make an interesting and valid point. I.e., does the phrase "pause until the front applications changes" mean:

(a) "pause until the front application changes to another application"; or
(b) "pause until there are significant changes inside the front application window."

The answer: it means the former. But if you want to perform the latter, it's not hard. Indeed, you can look for visual changes that happen in the front window. In fact, there are many ways to do that. This is one of my areas of expertise. If you tell me more about the exact nature of the changes, I can tell you which method you should use to check for changes.

4

Thanks for your reply. Is there ways to just have Keyboard Maestro look for major changes in the window like nested windows. I know there are image elements I can use to look for and how to get the name of front windows that change. However I was thinking things like nested windows that show up in windows at the bottom of the screen like in Pro Tools, Cubase, Logic, DaVinci Resolve etc. It may be all sorts of different nested windows and screen color might change so how would you go about just looking for a change in the front window layout. A screen playing audio would of course also be a change as images and audio wave forms pass by.

5

Since I don't have the apps you mentioned, I can't give specific advice for your case, but there are a variety of ways:

  • use Find Image to poll for certain images to appear;
  • use OCR to poll for certain words to appear (this works better on Mx chips than Intel chips);
  • use the Menu condition to poll for certain menu items to appear;
  • use the Button condition to poll for certain buttons to appear;
  • use AppleScript to poll if certain conditions are met; (but I'm a noob with AppleScript)
  • use the Execute Shell Script action to poll whether certain processes appear (sometimes apps will start sub-processes when certain windows are created.)

I have used most of these methods to detect whether something is on the screen. I have polling going on for 8 hours per day and it doesn't impact my overall CPU usage, so it's a viable method of screen/window detection.

6

That's interesting, I have never used this and will have to do some further research. Thanks for bringing this to my attention for use of OCR in applications.

I am a fan of this and just built one on this earlier this week when menu items go inactive. It seems like it would be less CPU intensive.

I don't really know much about this or how to create these so would also have to look into this. I have several Shell Scripts but only because I have pulled them from people or AI. Not really in the know on these yet.

That's cool, and very nice. Thanks for sharing the different methods with me, that is appreciated.

7

If you specifically choose the Apple Text Recognition, it works stupendously well. For example, there's a game I've played that frequently displayed the message "Click to continue." So I have a macro that constantly loops, looking for those words, (to click for me) and even though it's checking regularly, it doesn't slow down my game when I'm playing it. And it's even more amazing because the message "Click to continue" is displayed in a GIANT font (perhaps 200 pixels in height) and it still works well, and it's very fast.

I'm not really sure if the OCR code works in the neural engine (probably not, at least not on Intel Macs, since they don't have neural engines) or in a secondary processor ("performance core") which means the main processor isn't affected. I think it's a fair question for us to ask The Architect whether or not text recognition (or Find Image, for that matter) is running on a separate processor. I suspect that Find Image is running largely on the GPUs, not on the CPUs.

Since processors these days have multiple cores, it doesn't matter very much if the OCR code is running on a secondary processor. And even if it isn't, the real question is whether or not the code impacts your performance, which can be done only by measuring the impact. I've had several people tell me over the years that they don't like using OCR because it's "CPU intensive" but I tell them not to worry, because all that really matters is does it work properly and work fast enough?

/end opinion

8

I'd love to see some of this information added to the help file.
https://wiki.keyboardmaestro.com/action/OCR_Image

Oh my this is very cool and yes works way better than the English and was a night and day difference. I am excited to add this to some of my macros and this will solve some issues I have had in the past. It is amazing how many things Keyboard Maestro can do!

Thanks for the input on this and sharing your experience.

Yup good things to go by, I can see both points of view and great to be aware of what is slowing things down. Thanks for sharing all of that with me!

1 Like