New KM Trigger – Sound Effects!

I should get at least three claps for this fresh idea. I didn't include sample KM code because 99% of the work to get this to work occurs OUTSIDE Keyboard Maestro, so there isn't anything to share in terms of KM code.

With the new Shortcuts App in iOS (and I presume iPadOS, which I haven't tested yet) it's possible to auto-detect 15 different kinds of sounds in your environment and trigger a Keyboard Maestro macro if the specified sound is detected. For this to work, you need to have an iOS or iPadOS device sitting somewhere where you want the sound monitored. I usually carry my iPhone with me when I leave home, (so that wouldn't be to effective running it on my iPhone,) but my iPad is usually sitting at home so this would work best on my iPad, which can monitor the sounds in my house while I'm away.

So let me give you a practical example. Let's say you have left your house. You are carrying your iPhone but you have left your iPad at home. You can configure your iPad to automatically detect the sound of "Breaking Glass" using the shortcuts app. From this point, you could easily have the Shortcuts app send you a message. Technically, you can do this inside the Shortcuts app itself, using a Shortcuts action, but I found that the Shortcuts app refuses to send a text message to the same account, so you would probably have to use email to work around that "bug". But instead you can send a signal to Keyboard Maestro, and the method I have chosen to send a signal to KM is using the Remote Shell action in Shortcuts which uses the osascript command to trigger a KM macro. Shortcuts has an action called "Run Script over SSH" which lets you trigger a KM macro on your computer using the command such as:

osascript -e 'tell application "Keyboard Maestro Engine" to do script "96BD9CF3-...-33B59E8B9A0C"'

If you do this, you can use KM to send you an SMS text message such as "Breaking Glass Detected!" This way you will know if someone is breaking into your house (or if an animal ran through your front window.)

I have already used this method to get KM to send me a text message saying "Bless You" whenever I cough. (iOS comes with cough detection, but no sneeze detection. I presume sneezing triggers the cough trigger, but I can't "sneeze on demand" to test that. I noticed that clearing my throat always triggers the iOS cough detection algorithm.)

The 15 triggers you can get Shortcuts to trigger on (found inside the Preferences / Accessibility / Sound Recognition pane) are as follows:

  1. Fire
  2. Siren
  3. Smoke
  4. Cat
  5. Dog
  6. Appliances
  7. Car Horn
  8. Door Bell
  9. Door Knock
  10. Glass Breaking
  11. Kettle
  12. Water Running
  13. Baby Crying
  14. Coughing
  15. Shouting

So basically you can get a KM macro to trigger on demand if any of the above 15 sounds are detected in the area monitored by your spare iOS device.

More examples of how to use this idea:

  • Send yourself a text message saying "Door Knock" if you are in the back yard of your house around the pool, which would require one iOS device near your door listening for the knock, and another with you to give you the alert that your door is ringing or knocking. (I'm not rich enough to have a pool or back yard, so I can't test that.)
  • Have an alarm generated on the phone near your bed if an iOS device near your baby's crib hears a baby crying.
  • Send yourself a text message if there's ever shouting going on in your house. Perhaps that could be a thief, or just a party getting out of hand.
  • Send yourself a text message if a dog is barking in your house.
  • Send yourself a text message if there is a sound of a siren or the sound of fire in your house.

I neglected to mention you have to turn on "Remote Login" in your "System Preferences / Sharing" pane. And inside the "Run Script over SSH" action in Shortcuts you have to enter your Mac's IP address, and probably also a valid username/password.

Unfortunately Apple does not yet allow you to run this Shortcut from the Apple HomePod itself. That's a shame, as it has excellent microphones that would be perfect for this application. Hopefully they will add this feature next year.

Although I did explain everything, I suppose someone could create a step by step guide. I can't do that because I can't remember if there's some sort of security permission that I changed and can't repeat.

6 Likes

I sneezed today and the iOS cough detector was triggered.

6 Likes

Bless you!

2 Likes

Quite a novel idea and one I likely would never have thought of. Thanks for sharing this! It makes me wish I had more devices I could just leave lying around the house now though haha.

Thanks. It probably works best in a silent house... when you are away. If I'm at home watching a Youtube video my iPhone (& iPad) usually pops up every few minutes with a false alarm, like "A sound has been recognized that may be breaking glass" or "A sound has been recognized that may be a cat." So iOS is creating some false positives, but I haven't seen any false negatives yet.

1 Like

Great idea, and I'd love to try it for my own use case...but I can't find the detectable sounds in MacOS or on iOS...

The former I'm running Monterey. The latter I'm running 14.8. Perhaps I need to update the iPhone?

In iOS it's under Preferences / Accessibility / Sound Recognition / Sounds.

But I admit after a couple of months I turned this feature off due to too many false alarms. I think I should consider it for detecting suspicious events (Fire, Siren, Door Knock, Glass Breaking, Coughing) when I'm out of my home, and have it send me a text message, but with COVID rules "leaving the house" doesn't happen very often.

Genius! I was looking for preferences within Shortcuts. Thank you - off to give this a try!

Ok, so I can get my phone to alert me when a siren goes off...but I don't see how to have shortcuts trigger on the detected sound. Presumably this would be achieved through Automations within the Shortcuts app...but there's nothing obviously related to Sound Recognition or even "when i get such-and-such notification"...I'm sure I'm missing something obvious?

Great question. By now, I have forgotten myself. So I reread my original post. It isn't clear there. So I researched it again. Here's how you do it.

  1. Open the shortcuts app.
  2. Click on the Automation tab.
  3. Create personal Automation.
  4. Scroll down to the bottom and click on Sound Recognition.
  5. Click on Choose.
  6. Click on the sound that you want (eg, Fire Alarm).
  7. Click upon Done
  8. Click on Next.
  9. Click on Add Action.
  10. Add the action "Run Script over SSH" (it may ask you to change some System Preferences to allow this)
  11. Enter the "osascript" command that I describe in my first post, above.

I hope these instructions help. Let me know if it doesn't work, or even if it does.

Steps 10 & 11 have to do with informing KM on your Mac about the trigger, which you may not have been asking about, but I mentioned it here.