Is there any way to programmatically interact with a Chrome browser extension (e.g. clicking buttons and changing the parameters below)? Or am I stick with Click at Found Image?
I thnk you can use AppleScript/Javascript to select elements, you might have some luck searching for threads about selecting buttons in websites
Nyet. Extensions are designed to be isolated from content and are not externally scriptable by design.
2 Likes
I may not understand what's wanted. But I have automated opening the Bitwarden extension (Cmd-Shift-Y) and retrieving passwords via keyboard. I just now opened it and navigated through the dialog pictured above. The, er, key to moving around is the tab key. "Click" the selected item with the Return key.
Edit: it's possible that Cmd-Shift-Y is a new feature after the post I'm replying to.
The question was:
Manipulating the macOS UI is a different kettle of fish.
? The extension the question illustrates is the Bitwarden Chrome browser extension.
Running on the macOS.
OK -- I don't see that mentioned in the OP, but I am running MacOS.
This is going to be hard to do.
I had a quick look at your problem using Fiddlesoft's UI Browser. Dan made a tutorial on how to use it years ago
Anyway so step one is to actually click the extension:
activate application "Chrome Browser"
tell application "System Events"
tell process "Chrome Browser"
-- this assumes that Bitwarden is the first icon in the toolbar
click pop up button 1 of toolbar 1 of group 1 of group 1 of group 1 of group 1 of window 1
end tell
end tell
Where the wheels fall off is interacting with the popup. It's stacked full of elements which is bound to change when you update Bitwarden, so doing it via Applescript is out.
What you can do, is use the tab key( simulate a key press ) to get around the ui like a user would - and that's where KBM shines. Looks like the Bitwarden folk have made it all keyboard accessible, so you'll be able to script it.
1 Like