There is a fascinating new feature of macOS Catalina called Sidecar which supports using an iPad as an extension to macOS. This opens a new dimension for KM because it will almost feel as though KM is running on iPadOS because (some) presses on the iPad will trigger KM macros on your Mac! (see 31:20 in video)
However mouse clicks in macOS can't be used to trigger KM macros (not without changing the mouse driver, which won't apply to Sidecar.) And it appears that mouse click emulation may be limited to using the Apple Pencil (rather than a finger) and while that's okay (I could buy a Pencil) it seems that it may be limited to certain apps, and I'm not sure which apps. So the jury is still out on how this might impact KM. I'm inferring that movement of the Pencil on a Sidecar iPad would show up in KM as changes to MOUSEX() and MOUSEY(). That's definitely novel, if true.
However it does seem that the human finger will work on the bottom bar of Sidecar where the emulated TouchBar will appear, "even if you're running it on a Mac without a Touch Bar." If accurate, this means KM macros will be triggered by human fingers pressing emulated TouchBar buttons on the iPad. That would be novel. The keyboard is a device that can trigger macros initiated by a human finger, but this will be the first time that a finger press on a screen could trigger a KM macro.
There might even be a new API feature in macOS that KM would be allowed to access in order to detect "mouse clicks" from the Apple Pencil on the iPad. I'll bet that's true. And if so, I hope Peter takes the opportunity to hook into that API. I would imagine it could show up as a new trigger in KM. And if so, this would be the first time KM could trigger on primary mouse button clicks, even though it's just an emulated click.
Here’s a very informal three minutes screen recording of playing with Sidecar on a 12.9 inch iPad Pro. I’m not attempting to do any real study here, just putting my toes in the water and getting an initial impression to assess for myself if this is worth pursuing.
I didn’t annotated where I switched between Apple pencil and Trackpad control of the cursor tho if you watch closely I think you can see where the smooth movement and cursor jumping around the screen indicates the input method switching.
There’s a big difference between using the iPad as a separate monitor or in mirroring the Mac mode. In mirror mode you don’t need to reset your Mac screen arrangement as it’s just mirrored on the iPad (Captain obvious, right?). However, this does change the screen ratio of the Mac to something I’m guessing somehow matches the iPad as a good 1/3 of my 27” screen is converted to black sidebars reminiscent of watching a 4:3 formatted TV show on a widescreen monitor. Going the iPad as a separate screen route requires dragging windows and palettes into the iPad space and then resetting them back on the Mac when the iPad is disconnected. Not the smoothest arrangement or something I'd want to do multiple times a day.
Moving the cursor and clicking palette items works very well using a Magic trackpad. While you can use an Apple Pencil to activate palette items by moving and clicking the cursor, the Apple Pencil is not nearly as smooth or rapid as the trackpad and I would guess a mouse.
While normal iPad gestures continue to work in Sidecar, the cursor is not activated by finger movements making Sidecar a hand, keyboard, and trackpad/mouse mixed interface. While novel and interesting, it’ll take some time to see if I can work this into something practical that’s worth adapting to. Another project there’s not enough time for… Sigh.