How can I find text in an app and click on it?

In using the Notion app, I want a keyboard shortcut to add a new record to a database. Unfortunately, there is no menu item to do this. Sometimes, a button occurs on the page. I am able to find this and click on it using the Found Image action.

However, at other times, there is no button. The program simply displays the word, “+ New.” When I click on that manually, it creates a new record. What I really need is a Found Text function that is similar to the Found Image function. Is there a workaround or am I just missing something obvious?

I am not aware of any “found text” actions as I have attempted to do something similar in the past. Also, I have a sneaking suspicion that that link is not text anyway but another image.

However, there's a lot of people far smarter than me on this forum so I'm sure they'll be along shortly to help more haha.

I thought it might be an image, too. In fact, I took a screenshot of it and then tried a Found Image action to find it. It doesn’t work, so I am assuming it is text.

Even if it was text the found image should work on it. It probably didn’t work because it’s very light gray text on a white background. You could try adjusting the uniqueness (I can’t remember the exact term off the top of my head but it’s the sliding scale next to the image well) to see if KM will find it but it might confuse it with other areas of the screen that way.

Oddly, the Found Image action seems to have stopped working. It isn't even working with the blue button as it was previously.

If I recall, Notion is an application built on the ElectronJS framework, which create applications that are very pretty to look at, but suck the memory out of your system, and sadly don't have any accessibility hooks into their UI. Like, zero. Using the Accessibility Inspector that comes with XCode and pointing it at various UI elements in Atom—a well-known text editor, also built on the Electron framework—doesn't even recognise the text editing field (the main portion of the application's interface) as a text editing field.

So, purely thinking about this from a scripting point-of-view, the outlook appears bleak. But that's not to say there isn't some other means of achieving what you want, such as with the Found Image action, or something of similar ilk.

Frustratingly, these "Electron" apps are basically webpages, constructed with HTML, CSS and JavaScript, and you can even inspect its DOM if you press ⟨Cmd⟩+⟨I⟩ (or right-click, and select Inspect Element). They run off a Chrome browser instance. But, I can't see a way to send JavaScript commands to the application from an outside source, like Keyboard Maestro or AppleScript.

Do update us if you find any solution to this whatsoever, of any variety. It'll be very useful knowledge.

Yes, it is. But isn't Slack built on that framework, too?

Is it ? I didn't know that. The developers put a lot of effort in making Slack accessible and scriptable. Most don't, and I personally feel this is more the case with Electron apps because it's a good framework for a novice developer to build a fully-functioning application without much developer knowhow - as long as you know some rough HTML/CSS/JS, then the minutiae—such as accessibility—can be forgotten about.

Perhaps I'm being cynical. But with respect to Notion, my very brief stint with it didn't leave an impression of something that would be great for automation. But that was a while ago, so maybe it's worth getting the Accessibility Inspector out and having a whizz round the UI elements to see if anything is registered. If it is, and specifically, if you can identify that text element with it, then you'll be able to GUI script it in AppleScript.

They are working on API integration now. Some people I respect who have seen what they are doing say that it is going to be very robust. Thanks.

1 Like