"Type To Siri"

nok · March 28, 2019, 8:27pm

This is a Macro that I wrote to control HomeKit with Siri using KM or other Scripts. Currently there is no direct way to control HomeKit with scripting (Mojave did introduce a Siri SDK "Sirikit", that Peter is now aware of, see: Can Siri Be Used to Trigger a KM Macro?).

This Macro utilizes Mac OS Mojave's "Type To Siri" feature in which you can activate a Siri prompt and type in text to ask Siri for information or for controlling HomeKit devices.

PREREQUISITES:

First you must enable "Type To Siri" in System Preferences / Accessibility / Siri / "Enable Type To Siri"

Screen Shot 2019-03-26 at 1.12.21 PM.jpg1560×1176 214 KB
You may then modify how you 'activate' Type To Siri in System Preferences / Siri

Screen Shot 2019-03-26 at 1.59.31 PM.jpg1560×888 198 KB

NOTE: The default "Type To Siri" activation is "Hold Command Space", my macro uses "Hold Option Space", but you can change this to what you like in System Preferences, but be sure to then also change the Keystroke Action in the Macro.

The Macro runs some UI Scripts via AppleScript - which can sometimes be unreliable. I've tried to make them as reliable as possible while also adding some simple error notifications if something doesn't work. Problems may occur if you're running this on a Mac that's receiving lots of Notifications at the same time that the macro is run. As long as the "Type To Siri" notification window is the 'last' notification window shown, then it should work fine. I could fix this so that window order doesn't matter, but it wasn't necessary for my purposes as I have this running on a headless Mac Mini that's not receiving other notifications.

USAGE EXAMPLES:

These are just some of the many things I execute with this macro.

I'm typically running remote KM Macros (from other Macs) that trigger this "Type To Siri" Macro on my Mac Mini. For instance, when I return to my Mac Pro after lunch and I move the mouse, the Lights and Speakers turn back on (if they were on previously).
I also have a Hard Drive Dock that I use for Backups. When it's time to run the backup, I have a KM macro use Type To Siri to turn on the Wifi / HomeKit switch that the Dock is connected to, and then the backup runs. When the backup finishes another KM Macro runs and checks to make sure the Backup Volume has been unmounted and then runs another Type To Siri command to turn the HomeKit switch off, thus turning the Dock off.

USAGE:

Set the TypeToSiriQuery variable to a Siri query like "Turn Off Studio Light" etc.

image.png1270×948 93.1 KB
Run the Type To Siri Macro

image.png1270×6438 1.06 MB

Type To Siri.kmmacros (52.6 KB)

Version 1.2 2019-03-28

Improved AppleScript code to 'find' the correct Notification Window for Type to Siri instead of having the window 'set' to 'first' or 'last' etc. This should make the macro a bit more reliable when there happens to be multiple notifications on screen.
As a result of 1. above, I have removed the 'repeat this action 2 times' because it should not be necessary any longer.

Sleepy · March 26, 2019, 9:36pm

Wow. Just wow. Did I say wow yet? That's 20 times more impressive than my primitive method for addressing HomeKit control. Your post is informative and innovative. Brilliant. AppleScript always makes me struggle so I'm really impressed. Very inspiring. I will study this. That's a whole different approach that I should have come up with myself... but didn't. Throw my old ideas away. I wonder what else can be done with this approach. I haven't even loaded your macro yet and I can see it's a winner. But it would sure be nice if the next release of KM could support this a little more directly with an action. After I study your solution I may propose a new action. But I hear it's on Peter's future feature list, so there's no point in pontificating.

JMichaelTX · March 27, 2019, 2:35am

Thanks for sharing. This looks like a very cool & useful macro.
I'm looking forward to upgrading to Mojave so I can take advantage of features like this.

peternlewis · March 28, 2019, 6:38am

Very cool!

You might like to use the parameter facility in the Execute a Macro action to pass your parameter instead of passing in a variable. You can always just start your macro with:

Set Variable TypeToSiriQuery to %TriggerValue%

to leave your macro unchanged.

nok · March 28, 2019, 8:33pm

Updated to Version 1.2

nok · March 28, 2019, 8:33pm

Thanks Peter, that's an interesting idea. Is there any other benefit besides the fact that I'll be able to remove the "Set Variable" action? I suppose I'll actually be moving & modifying the "Set Variable" action into the Execute Macro Action.

peternlewis · March 29, 2019, 3:20am

It just means that when executing the macro you don't need to remember the name of the input variable.

Scott721 · November 16, 2020, 6:56am

All - I must be missing something or this setup process is missing a step. When creating the macro as shown above the "Type To Siri" selection does not appear in the dropdown menu of "All Macros" to choose from when selecting the "Execute Macro" step.

In the attached screenshot you can see that there is no "Type To Siri" macro. Did I somehow miss a step to create a macro called "Type To Siri" first? If that macro does not exist the subsequent steps cannot be followed.

I get the rest of it and it all looks logical to me but I can't build this unless I can get past this step. Can someone assist?

Thank you.

rolian · July 6, 2020, 9:35pm

Seems like you missed downloading the Type to Siri macro at the end of the initial post?

Scott721 · July 6, 2020, 9:41pm

Thank you. I did miss that. My apologies for not seeing that at the end of the post. I’m very familiar with the use of AppleScript and was a former Quickeys user (remember that app?). Using forums for help is unusual for me and I simply overlooked the downloadable macro.

I appreciate the help from the community and the quick response. Now it’s back to HomeKit and KM. I’m going to attempt to connect my iMac to my desk lamp (plugged into a Wemo smart switch and connected to my HomeKit through a HomePod as a hub) so that when the screensaver comes on (or the computer locks) it issues a command to Siri for it to “turn off the desk lamp.”

Thanks to @rolian and any others that may have read my post.

SuperIngo · September 2, 2020, 5:49pm

Thank you for the Type To Siri macro!
I had to change the string for the process name to work with my German MacOS installation. I am sure this is needed for other non-US installations too.

After replacing the following in all Script blocks it worked fine for me:

tell process "Notification Center" -> tell process "Mitteilungszentrale"

The strings for the description of the notifyWindow and for the textField don't need to be localised.

JoelBerger · November 15, 2020, 8:59pm

Does anyone have this macro working with Big Sur? It appears the AppleScript for interacting with Siri has changed and I'm not sure how to figure out what the equivalent elements should be. Any help is appreciated!

stephenll · November 20, 2020, 1:10am

Did you make any progress trying to get this to work? All dead ends for me.

JoelBerger · November 20, 2020, 3:43pm

Unfortunately, no. I tried messing with UI Browser but quickly got in over my head.

I believe typing to Siri can be accomplished, but capturing the response is where I'm not finding a starting point let alone a path forward.

nok · November 20, 2020, 5:51pm

Sorry guys! I would love to help, but I'm not sure when I'll be installing Big Sur.

stephenll · November 24, 2020, 9:47pm

Here is my simple and very not robust temporary fix...

I am using KM to simulate what I would type if I were using Type to Siri.

crabl · February 17, 2021, 4:53pm

Here's an updated version of the AppleScript that will run on Big Sur:

tell application "System Events" to tell the front menu bar of process "SystemUIServer"
	tell (first menu bar item whose description is "Siri")
		perform action "AXPress"
	end tell
end tell

delay 1

tell application "System Events" to tell the process "NotificationCenter"
	set notifyWindows to every window
	repeat with notifyWindow in notifyWindows
		if description of notifyWindow is "system dialog" then
			set elements to entire contents of notifyWindow
			tell (first text field of first group of first UI element of first scroll area of notifyWindow)
				set focused to true
				set value to ASTypeToSiriQuery
				perform action "AXConfirm"
			end tell
			exit repeat
		end if
	end repeat
end tell

Edit: I should mention, I'm not very good with AppleScript, so this is probably going to break in the future, but it works for now.

"Type To Siri"

Options