Automating Safari Google Sheet *Extension* interactions?

Hi all,

I'm completely fresh to Keyboard Maestro so forgive me if this is simple.

I set up a basic macro which works well, but of course requires taking over my mouse and clicking on a very specific area of the screen.

What I'd like to do is replicate this behaviour but within the browser, so I am not reliant on KM taking control of my mouse. What I'm unsure of is:
a) Is this possible, so that I can either keep the window in the background or minimised, and have the macro run throughout the day?
b) I need to interact (click) a Google Sheet extension - will this work okay?

Highlighted in blue is the Google Sheet extension - it pops out in a side window, and the Inspect window is able to pick up the various elements so I hope there's a way I can target the clicks via Xpath somehow?

Thanks very much in advance!

I'm not good at AppleScript, so I'm afraid this is not a very good answer, but there's a small chance that AppleScript can send clicks to apps without "taking control of the mouse." We will need an AppleScript Aficionado to help you here. It might even be able to do this when a window is minimized. AppleScript can be run from inside KM or from inside other things like Automator or Command Line cron processes.

I'm not sure if your question "b" is referring to a separate issue or if it's contained within the restrictions of "not taking control of the mouse." If the latter, then no, it won't leave the mouse untouched. If the former, then yes, the approach could work.

I know this isn't the best answer you could have asked for, but nobody else has spoken up yet, so I thought I'd try.

1 Like

Thank you—I hope there's a way too!

What I was referring to with b) is because it's a Google Sheet Extension I am trying to automate the clicks on, I don't know if things are handled differently. I think it technically loads like an iFrame.

And while I can copy the Xpath, and find it okay within the browser:

When testing a Macro that attempts to click that using Xpath (I found a template on the forum here, although it's qutie old), it can't seem to find it:

Can you show the KM action that does the click? If you were using the KM Mouse Click action, you wouldn't get that error.

I used this template from the forum which uses javascript to click.

Aha! The phrase "using a macro to click the button," made me think you were using a different kind of action. Since you are using a JavaScript program to click the button, I can't help. Sorry.