I am using an app named Vivix that connects keyboard shortcuts to OpenAI (using the user's OpenAI key). When a user-defined shortcut is pressed, a user-defined prompt is applied to the clipboard. The output can be set to go to the clipboard, added to text, replace text, or a modal box. It feels very similar to basic Keyboard Maestro use.
Are there any plans to add this functionality?
I'm sure "The Architect" is looking into whether macOS Sequoia has a public API that permits this functionality. He has said if he can find an API for it, he would like to add it. So really, the question is for Apple to address... will they have an API for it.
However I'd like to point out that Siri on macOS has a "Type to Siri" feature already, and while this feature does not have a public API, it's possible to use KM to submit questions to Siri, and it's possible, but difficult, to get the answers back. During WWDC Apple said it would introduce "Type to Siri" to iOS, but that doesn't mean they will introduce an API for it for either macOS or iOS.
From reading your question, it appears that you are interested only in text results. But AI often includes images or audio, and maybe video. Are you interested in those, and if so, how would you like KM to return those results, since KM variables and clipboards can't store audio or video?
Over the last few days Apple has said it is not rolling out all its AI features immediately, and it won't be rolling out its AI features worldwide (e.g., not in Europe.) If Apple does roll out an API for AI, it may not arrive for six months, and then it might take a few months before KM can be updated to use it. And then it may only be usable in countries where the API works.
Lastly, macOS Sequoia has three levels of AI. The first level works locally. The second level sends requests to special computers on the Internet called "Private Cloud Compute." The third level sends data to OpenAI, but only if the user manually approves it. I presume you're not asking if that third level can be automated with a KM macro. Or are you?
I was thinking that this did not have to depend on Apple Intelligence. The tool I am already doing this with is called Vidix, and today I discovered that there are others just like it. In fact, BBEdit does something similar, where ChatGPT can be executed on text in a worksheet. The user must supply an OpenAI key for Vidix, BBEdit, and similar tools that allow direct access to GPT-4o. This functionality could simply be an additional action available to be used in a KM macro. It is very much opt-in, because the user would have to have an OpenAI account and they would have to generate a key and enter it to use the feature.
Here are a couple of the macros I am using out of about 30.
shift-option-cmd-R
Write a response to the text below that is 140 characters or less that offers an agreeable statement that represents support and agreement with the text, and adds some additional information as well. Go right into the response without starting with a word such as “absolutely,” or any similar word.
shift-ctrl-cmd-R
Write a response to the text below that is 280 characters or less that offers an agreeable statement that represents support and agreement with the text, and adds some additional information as well. Go right into the response without starting with a word such as “absolutely,” or any similar word.
Search the forum here and you'll find someone posted a subroutine to query ChatGPT with a key and get the results back into Keyboard Maestro; maybe that'll get you started?
-rob.
Thank you.
I gave a quick search and came up blank, but I know it's here—I think I even replied in it, but now I can't find it :(.
-rob.
Now I see what you mean. I guess that's possible, but it will become obsolete as soon as Sequoia comes out, because Sequoia users will have access to GPT-4o without having to do any of that key registration. It will be baked into macOS. So there isn't much point in doing something that's about to become obsolete.
Moreover, Sequoia is also supposed to be able to handle the registration process for a paid GPT account too.
These tools may still be useful for people who use Intel Macs.
That's an excellent point. I bought an M1 Mac the day it came out, so I tend to forget about Intel. Sorry.
BetterTouchTool has a built in action that returns the GPT output from the text you have selected as a prompt. You could use it from there, or call it from within a Keyboard Maestro Macro
Not really -- as you mentioned above, Apple said many times throughout WWDC that much of Apple Intelligence won't be present when Sequoia is released, and when features do arrive they should be considered to be in beta.
Not that I'm advocating for a special KM action. If other apps can do this sort of thing, use KM to drive them! For those who really want to get into the weeds, KM has an "Execute a shell script" action, the shell has curl, and ChatGPT has a curlable API...
This just occurred to me this morning. Why would I want to have this, other than to save space on the menu bar and save a few dollars on an extra utility?
I am going to look into trying the shell script method, so thank you for that idea.
For now, though, I can use Simulate Keystroke, so I can type easier to remember typed strings to trigger an obscure keyboard shortcut from the other app.
@peternlewis any thoughts? Having AI build my macros would be a godsend!
Peter already indicated on another thread that if there's no public API for it, he can't implement it. It's not really a question of wanting it, it's a question of whether Apple provides the interface to access it.
Even when Peter implemented Apple's OCR into KM, I recall it looks months after the release of it before it made its way to KM.