Each Found Image Instance?

I want to click relative to a found image then click relative to the next instance of that same image.

As an example:


The goal is to click each circle from top to bottom, then scroll to check for more. The logical way to do this seems to be:

  • Click 130px to the right of the topmost image of a square.
  • Click 130px to the right of the second topmost image of a square (this is what I don't know how to do!).
  • Repeat until no more complete square images are found.
  • Scroll down a bit.
  • If more square images are found, continue clicking circles.
Expand for a hacky idea that doesn't work...

This probably doesn't work because there's zero fuzziness, and besides, there's probably a simpler solution I've overlooked, hence this request for help... Anyway here's my failed experiement:

  • Click the top circle relative to the top square.
  • Capture an area around the mouse to get an image of the circle to a Named Clipboard.
  • Scroll down 30px and capture the same size area but 30px higher than before to the System Clipboard.
  • Repeat scroll and capture until the Named Clipboard and System Clipboard no longer match.

Why not use a loop over a set of found images?

KM 0 2022-06-26_17-05-26

  1. Put the image of the square you want to find in the image well.
  2. Make sure you choose "Top to Bottom"

In the actions you can then click 130 pixels to the right of each occurrence.

The other requirement - scrolling to see if there are more - is a lot harder but this suggestion will at least get you started.


That's perfect! Can't believe I didn't think of images as a collection. First time I've come across it. Brilliant! Thanks!

1 Like

Figured that part out:

Scroll Until Image Off-Screen.kmmacros (25.2 KB)



Well @noisneil - that’s a very clever collection of actions you’ve put together there. I wish I could’ve come up with that! :clap:

1 Like

I've added the action group above in case you wanna try it out. :slight_smile:

You could of course also use a found image's location and dimensions rather than static values around the mouse. Works well for me!

Thanks - I'll keep that in my toolbox!

1 Like
1 Like

@tiffle Any idea if there's a way to detect that you've reached the end of a document and there's nowhere left to scroll?

You know - I was wondering how you knew when you've reached the bottom (easy for this smart-ass to say now, isn't it!)

I can think of 3 ways to do this:

  1. (and I've actually used this) is when there's something at the bottom you can recognise. This was easy in my case when scrolling down some web pages and some word processing docs - there was a footer at the bottom that I could detect. If it wasn't detected then I wasn't at the bottom! I suspect you don't have this luxury.

  2. Detect when the scroll bar reaches the bottom. Here's an example of a scroll bar that yet has some way to go:

KM 0 2022-06-27_15-53-40

  1. And this is a long shot specifically for your case: when you click on those circles does something change? Like, for example, a check/tick mark appears in the circle? Or something gets emboldened? If it did then you'd know you couldn't scroll any more when you keep seeing the same thing after trying to scroll further. Maybe the fact that the circles are different colours could be used to indicate an attempt at scrolling didn't do anything since the circle of that last, specific colour is still in the same place?

As I say - I've only ever used the first of these but I don't think that will help you specifically.

What do you think?

Ideally, I'd like this to work for any app that scrolls. The squares and circles thing was something I mocked up in Photoshop to demonstrate the concept; it's not an actual app.

I've seen a few scripts that scroll until the window is displaying the bottom of the document height (I'm probably getting the terminology wrong), so I wonder if a script condition might be the thing. I'm too much of a noob to even investigate that unfortunately. Most of these scripts seem to relate to browsers, but perhaps there's a way.

This seems the best current option for sure. Works nicely actually!

Scroll Until Bottom - Scroll Bar.kmmacros (31 KB)

Macro screenshot

This won't work for windows in windows that don't have scrollbars, of course.

If the window is AppleScriptable then the scroll bar takes on a range of values from 0 (top) to 1 (bottom) which would be both read/write-able. So, for example, halfway would be 0.5. That’s something you could explore with UI Browser.

Yes, I’ve even contributed one in the past. But that depends on Javascript in the browser and so only works there…

I had a look in UI Browser, but it doesn't seem to see the scroll bar. Google didn't throw up anything useful related to getting the current scrollbar position with AS. I have a feeling it would slow things down quite a lot, but I suppose it would be a bit more dependable than found images.

When I google "AppleScript scroll bar" I get lots of hits. Here's one:

That's great on a per-app basis, but it's not universal. Still better than nothing I suppose!

don't hold your breath...

1 Like

This is confusing as all heckery to look at... but it works.

Click All Images (Scrolling).kmmacros (72 KB)

Macro screenshot

CleanShot 2022-06-28 at 00.21.14

1 Like

Yeah - that’s manic :face_with_peeking_eye:

1 Like