Click at Found Image in AREA of Front Window

I couldn't find this anywhere else, and it's quite handy. It will find and click an image in a specified area of the Front Window, regardless of its position on the screen.

Here's a demonstration of a highlighted area moving along with a window, to give you an idea of what I mean.

Demo Video

demo


These actions are required for macros 2 & 3:

Click at Found Image in AREA of Front Window.kmactions (3.1 KB)

Screenshot

Click at Found Image in AREA of Screenset.kmactions (1.4 KB)

Screenshot


Place the following macros in a group that will be available in all applications.


ALL MACROS:

Screenshot and Area Macros.kmmacros (227.6 KB)


INDIVIDUAL MACROS:

Macro 1:

KM - Set Area Coordinates for Existing Action.kmmacros (55 KB)

Macro screenshot

Demo Video

area

Info

Setup:

Set two hotkeys and enter the value of one of them into the box that currently says "Front". This hotkey will use area relative to the front window and the other will use area of the screen.

Use:

Trigger this macro and then click at the top-left and bottom right of the area whose coordinates you wish to capture. Next, click in the first coordinates field of the relevant action and the values will be entered.

Macro 2:

KM - Screenshot & Custom Area.kmmacros (65 KB)

Macro screenshot

Demo Video

custom

Info

This macro requires the following two actions to be saved as favourites:

Click at Found Image in AREA of Front Window

Click at Found Image in AREA of Screenset

Setup:

Set two hotkeys and enter the value of one of them into the box that currently says "Front". This hotkey will use area relative to the front window and the other will use area of the screen.

Use:

Triggering this macro will trigger the screenshot utility. After you click and drag over the area you'd like to capture, you will be prompted to capture the search area. Click at the top-left and bottom right of the area whose coordinates you wish to capture. An action group will be automatically created and populated with the screenshot and appropriate coordinates.

Macro 3:

KM - Screenshot & Area (Auto).kmmacros (63 KB)

Macro screenshot

Demo Video

auto

Info

Info:

This macro uses the submacro KM - Area - Start Screenshot (SUB!) and requires the following two actions to be saved as favourites:

Click at Found Image in AREA of Front Window

Click at Found Image in AREA of Screenset

Setup:

In the purple action below, set the number of pixels to search beyond the bounds of the screenshot you willl take.

Set two hotkeys and enter the value of one of them into the box that currently says "Front". This hotkey will use area relative to the front window and the other will use area of the screen.

Use:

Triggering this macro will trigger the screenshot utility. After you click and drag over the area you'd like to capture, an action group will be automatically created and populated with the screenshot and appropriate coordinates.

This submacro is required by macro 3:

KM - Start Screenshot (SUB-).kmmacros (46 KB)

Macro screenshot

2 Likes

Hi @noisneil
Recently I have been playing around with making new Macros and Actions using XML/AppleScript. It all started with looking up how to make a Macro to make new Text Snippets. Basically it is possible to create a new Action in a Macro with its Parameters set to Variables that you have set up.

So, for the section of your Macro that looks like this…

image

… and which requires you to click in the field you have called CLICK ME, I was wondering if a brand new Click at Found Image Action could just be added to the end of the Macro, ready for cutting and pasting where you want, with the area coordinate already filled in using the values your Macro has obtained.

So the Macro below demonstrates how four Variables can be used to create a new Click at Found Image Action pre-populated with those numbers. It is based on AppleScript by @unlocked2412

The Example uses a Prompt to get the 4 Values, but in your case you would just feed in the four Values you got using your Macro.

EXAMPLE Create Click at Found Image with Custom Area from Variables.kmmacros (6.7 KB)

Click to Show Image of Macro

Click to Show Gif

Screen Recording 2022-03-21 at 17.10.21-Animated GIFF 640 12fps

Using this method to create Actions and Macros from Variables is very powerful and can be adapted to lots of other uses. The original AppleScript came from here:

1 Like

Ooh, that's nice!!!

I tried it and it's really smooth! The only snag is that I need the rest of my action group to also be created around the click at found image action for it to work. Is that possible? Can I load the whole group as a favourite and then populate the fields? I'm not confident enough (yet) to tinker with your example code.

I was hoping you could just bolt as is onto your current Macro and then copy and paste the resultant Action into other Macros. But it that isn't the case and as you have your Macro already working it might not be worth it. Just something to consider for the future, if you want to make Macros that in effect make other Macros. The problem with Keyboard Maestro is that it is endless in its possibilities and we are all always learning new ways to do things... :rofl:

1 Like

Hi @noisneil, my macro starts with a favorite action (red). KM 10.0.2 will then fade out and pause.
I can now select an app (here Finder) and take my screenshot after pressing the ⌃ key.
The screenshot is inserted and KM is minimized again to select the area for the screenshot.

2022_03_21_Support_1

07)ImCoord. <A4F3 220321T203408>.kmmacros (36,2 KB)

,Click to show image

If the screenshot should not be saved in the KM Editor, I use this macro

2022_03_21_Support_2

01)ExCoord. <3345 220321T203351>.kmmacros (52,3 KB)

Click to show image

Here is the macro for the displayed KM notification (center), which I have included in my macros.

04)HUD Display - ImShot <57F2 220321T203438>.kmmacros (9,0 KB)

Click to show image

04)HUD Display - ImShot <57F2 220321T203438>

The ⌃ key is on my German button here:

@appleianer That's so cool! It's kinda hard to tell from the video, but I think I have a slightly different objective here, which is to define a search area for the found image within a window. The window can then move around the screen but the search area follows it. I think that second set of macros (image to path and display image from path) is one I might have had a hand in. Looks so slick how you use it. I never use palettes as I have a Stream Deck, but you're making me wanna!

Ok so I found a way to do it! I just saved the Get FrontWindow Left and Top group as a favourite and loaded it before running your script! So neat!!

Screenshot

Do you happen to know how to use keyboard shortcuts to select the group that a selected action is in, in the KM editor? If I knew that, I'd be able to also group them and rename the group! I can use the up and down arrows to select different actions within a group, but how do I select the group itself?

@appleianer What's this?

Pasted_Image_21_03_2022__20_53

@noisneil this is my assigned name for the favorite action.

@noisneil I don't know if you understand you correctly. I select a macro group like this, then rename it:

2022_03_21_Support_5

01)Rename <3402 220321T222214>.kmmacros (46,9 KB)

Click to show image

01)Rename <3402 220321T222214>

Sorry, I don't understand. What is the favourite action? Is it just an empty Click at Found Image action?

I meant the group of actions in a macro. When one action is selected in a group, I can use the up and down arrows to move to the action above or below, but I don't know which keystrokes to use to select the actual group they're in.
Keyboard_Maestro_Editor_—_zz_TEST

@noisneil ok, you mean it like this:

2022_03_21_Support_1

Rename Group <8C7B 220321T232602>.kmmacros (18,9 KB)

Click to show image

Rename Group <8C7B 220321T232602>

No... I know how to rename a group.

I don't know how you make a gif like that... or upload it here... so I'll have to attach a video.
1.mov.zip (121.0 KB)

See how I'm using the arrows to move up and down? How do I select the group that holds the actions without using the mouse?

@noisneil I create a favorite action in which the first input field (here 0) is marked and later appears activated when used:

2022_03_21_Support_2

After I take a screenshot, KM is re-enabled and the repaet action (tab) then allows pasting it.

2022-03-21_23-53-15

Ah ok yeah thought so. Seriously, I'd like to know how you're making those gifs. They look so nice!

@noisneil sorry, just noticed that if you are in the group you can rename it like this:

2022_03_22_Support_3

Nooooooooo..... :joy:

I know how to rename a group! Haha. This is too funny!

I'm trying to SELECT the group without using the mouse, when one of the actions INSIDE the group is currently selected.

(Also, how are you doing the gifs?!)

1 Like

@noisneil thanks to ScreenFlow I can quickly convert my screen recordings into GIF's. Of course optimized/automated with KM :wink:

Video

But there are also good tools like Gif Brewery, 3 or Gifski with which you can create GIFs.

2 Likes

@noisneil but now... 2 x ⇧⇥ and then the rename action with a shortcut. This also works without a mouse.

2022_03_22_Support_2

1 Like

Thanks!

Get Area of Front Window (Auto).kmmacros (61.2 KB)

ScreenFlow

1 Like

When I add a favourite, it doesn't allow me to click in the field before saving, like you do in the gif. I'm guessing that's your own save favourite macro....?

I've ended up using a different method anyway and have edited the macro at the top of this thread.

Cheers for the suggestions! :pray:t3:

1 Like