Autoclick

Questions & Suggestions
1

Here’s a new feature idea that I think would make KM even more powerful, especially for repetitive web navigation tasks.

The idea:

Imagine being able to enable a temporary “autoclick mode”, where Keyboard Maestro watches the screen for a list of predefined images (e.g., buttons or links the user has captured), and clicks them as soon as they appear — all automatically, and within a bounded time window.

Example use case:

When logging in to some websites (like banks or other secure systems), there are often 3–5 buttons or links that have to be clicked in sequence: “Login”, “Continue”, “Verify”, etc. These UI elements are usually static and visually distinct. Setting up individual image-click actions works, but is a bit tedious for multi-step flows. And sometimes timing or animation makes it tricky.

How it might work:

The user configures autoclick with a set of screenshots of buttons from their banks’ websites. Then, when they go to log in:

  • The user activates autoclick

  • For, say, the next 10 seconds, KM watches the screen.

  • If any of the images appear, it clicks them — in any order — as they show up.

  • If nothing is clicked for 3–5 seconds, the mode exits automatically (or the user can hit Escape to cancel).

Why this would be useful:

  • It would make multi-step web navigation feel magical.

  • No need to hardcode timing or order — KM just clicks what it sees.

  • Great for handling semi-variable sequences of clicks (e.g., MFA screens, extra pop-ups, etc.).

Naming thoughts:

It could be called something like “SmartClick” or “ImageClick Sweep” or just “Autoclick”.

Thanks for considering it! Would love to hear if this might be feasible or if something similar already exists.

2

If I read your request correctly, something like how I've shown below should get you quite close.

Autoclick.kmmacros (20 KB)

Macro Image

Keyboard Maestro Export
Keyboard Maestro Export1202×3570 378 KB

But this is just a very crude demonstration. And it would continue to click the same image over and over if still present on the screen.

So maybe something like this would be better. It cycles through looking for images, once clicked it will not click that same image again.

Autoclick 2.kmmacros (25 KB)

Macro Image

Keyboard Maestro Export
Keyboard Maestro Export1202×5452 699 KB

3

You don’t need a “new” feature to accomplish what you are asking for. Keyboard Maestro already supports this functionality. You can implement an "autoclick mode" using built-in image detection actions or, more reliably, through JavaScript. The programmatic approach is often superior because it can simulate clicks or form submissions even if the page is not currently visible or in the foreground.

Click at Found Images:

Press a button action:

[…] There is a popup menu (v11.0+) to scan the front window of a selected application. Ensure the window of interest is the front window in your target application, then select the application in the popup menu. Keyboard Maestro will scan the window. Then select the popup menu and there will be an additional menu of all the buttons in the front window of the target application:

image
image934×440 51.6 KB

or what I use the most:

JavaScript to simulate clicks:

Add the Execute a JavaScript in Browser action. Even a one-liner usually works:

  1. document.querySelector('button#login')?.click();
2 Likes
4

This is actually pretty close! I created something like this awhile back, but didn’t realized you could stack the image searches that way (mine was while → if → click → else if → click → else if → click…. very clunky).

This isn’t really something my wife could (or would) set up, but for me it’s good. I also appreciate the SECONDS() thing, I would’ve been looking for a timeout somewhere and probably not found it.

Is there a convenient way to force it to stop if the user presses Escape? I’m nervous about accidentally pasting in an image that is too generic and having it go nuts clicking all over the screen…

I still think the autoclick idea would be a good future idea for less-experienced (or less-invested) users, e.g. like my wife or daughters.

5

Have you tried changing the "Until" Conditions to "any of the following" and adding a "Key condition" for the ESC key?

6

I do not know what is limiting your wife, but if it is having to set the images within the KM Editor you could also load the images into the If/Then from a file or clipboard.

You do this by choosing, for example, Named Clipboard from the drop-down menu when clicking the arrows next to "Image":

Illustration of set-up

image
image944×1446 167 KB

This way you can now deal with setting the screen grabs through a prompt, entirely outside the editor. For something like this I've made the following macro (that I have also combined with the "Autoclick" macro further down), to capture the screen captures and store them into separate Screen Grab "Wells":

Set Image Well to Screen Capture.kmmacros

(KM v11.0.4, 62 KB)

Macro Image

Keyboard Maestro Export
Keyboard Maestro Export746×9762 1.35 MB

I am particularly pleased with how this Prompt's features and the way it updates. It is set up with 5 predefined "wells" that screen grabs can simply be captured into. Or just as simply, be deleted with a change of the mode. The Prompt update check boxes to show which of the wells holds images, and by pressing the "Display" butten it will open a display, previewing the content of all the image wells.

Image of prompt

image
image1756×722 94.9 KB

I have also majorly overhauled the "Autoclick" macro, and combined it with the Set Image Well to Screen Capture from above. I am still not at all sure if it really makes sense with such a generalized tool for something as specialized as clicking UI elements in progression — something usually needing a level of granular tweaking. But as I enjoy building iterating macro structures like this I still built this macro, now having some level of granular settings, for how the macro moves through a set of stored images to find and click:

Autoclick 3, combined with "Set Image Well to Screen Capture" and "Cancel caller whenever escape key is pressed" (v11.0.4)

ΚΜ-forum – Sharing Macros.kmmacros (151 KB)

Macro Images

Keyboard Maestro Export
Keyboard Maestro Export784×4740 701 KB

Keyboard Maestro Export
Keyboard Maestro Export784×738 83.6 KB

Settings:

  1. Duration:
    Total duration, in seconds, until the macro stops looking for images to click.

  2. Click Quarantine:
    Quarantine time, in seconds, until a specific clicked image can be clicked again. Set to any non-numeric value (or more precisely non-calculable numeric expression, of course including numbers) – "X", "inf", "∞", etc. – will effectively set the quarantine to last indefinitely, making the clicked image not be clicked again until the end of the macro execution.

  3. Click Pause:
    Pause time, in seconds, after an image is clicked until any next image will be clicked, be it the same clicked image or another.

  4. Sequential:
    Set to "0" the clicks will be performed in any order; set to "1" the clicks will be performed in sequential order starting at #1; "2", sequential starting at #2 (wrapping around to click #1 after having clicked #5), etc.

As suggested by @Nige_S above, the "Autoclick's" Until is set up with an "Escape key is down" condition, but as this Until's loop is quite a long one, it takes a bit of time between each evaluation of the Until's condition. Therefore, I also bundled this macro with a tool – an asynchronously running Submacro – that monitors for escape being pressed, and if so, immediately cancels the caller.