Personally I’m finding myself more and more using compiled Swift scripts (i.e. binaries), also in conjunction with KM.
For my personal use this is not a problem, since I just throw them into my ~/bin folder.
But I noticed, especially while writing this post, that this might not be as straightforward to other users. You have to provide instructions on how to to place executables in the PATH, and furthermore how to make the PATH available to KM. — Or telling the user to prefix the executable with an explicit path in the shell script. Which isn’t very user-friendly either.
This is due to the fact that KM macros are plain XML files.
Now, let’s have a look at how this is handled by LaunchBar:
When distributing a LaunchBar action I can include any script, executable, etc. inside the package/bundle. As long as I refer to the correct relative path, of course.
This model (the package/bundle model) makes distribution so much easier, more attractive. Besides being able to put my Swift bins into the package I can also distribute entire perl modules and —if needed— a whole pandoc installation along with my Actions/Macros. For an example see here: page | direct download.
I know that this would mean a really major change to the KM macro format. But maybe a new, advanced macro format could coexist with the traditional one?
I use a lot of scripts in my KM Macros. For the most part, I just link to the script file, rather than including the text of the script in the Execute Script Action. Of course, this does not work well when I upload.
So, this might not work as smooth as your package/bundle idea, but maybe an easier method to implement is having an official KM folder for custom resources. Call it "bin", or "resources", or whatever makes sense, and put it the user's documents/Keyboard Maestro folder.
Then, when I needed the resource, I could just use the resource name without a path, or, if need be, a KM token for the folder. I like no path the best.
But, if I get you right, this would be a directory like for example ~/Library/Application Support/KM/Scripts or some user-defined directory like ~/Documents/Scripts/KM
This would still imply that we have to tell the user —in case we share our macro— where to install the files. Right?
Not my idea; just like LaunchBar does it —as an example. But it seems to me like a good compromise.
Another —far more sophisticated— approach would be to define an installer format, and then upon installation the installer puts the macro file in its location, the binaries in its locations, etc.
But that —while probably being “cleaner”— sounds way too complicated, and too unflexible. Hence, the bundle thingy, borrowed from LaunchBar.
I would prefer that it NOT be in ~Library/... because that is not easily searchable by Spotlight and other tools.
Well, it would unless @peternlewis modified the KM export/import file to include resources like you and I are discussing. So then, when I export a Macro with an Execute Script with a file without a path, KM would include that script in the export. Then when the import is done, it would put the script in the official KM resources folder.
And, as you said, KM would continue to support the plain XML export/import.
One benefit of this approach is that if only the resource was revised, then we could just upload it instead of the entire macro.
Related side question:
In MS Windows you could create a zip file with a relative path, so that when the receiver unzipped it, it would default to the same relative path on the user's machine.
Can this be done with macOS zip files?
If so, this would make export/import of resource files much easier.
I think we have posted at the same time. Have you seen my post above?
This is not a ZIP file (in the sense of an archive). This is rather an installer, either with the installer routines contained with itself, or in compliance with the OS that executes the routines. See my post above, concerning installers.
Well, I feared my post would start a discussion way more complicated than necessary.
So, my point is just this:
Personally I’m fine with KM’s plain XML files as they are. But I’m seeing myself not posting many macros that involve dependencies (like compiled scripts), even if they —maybe— would be worth to be posted. Just too much trouble to explain the PATH thingy each time.
The bundle approach —far from being a perfect solution— seems to me as a compromise. Not too complicated and —probably— capable of co-existing with the current plain format. Even Apple is using it, in various contexts.
I mean, if you install dependencies in certain folders (not being the same folder where the macro lives) chances are good that we end up in a mess. (Not we, but the occasional user). Is it any better with the Actions installed in the Actions folder by dropping the archive on the KM iccon in the Dock? And then, when the Action has to be updated the old one has to be removed before?
Yeah I meant that. Or even better, make a wiki post or Gist or something and link to that for users who are unfamiliar with how PATH works and how to use it.
You have to distribute the files along with the macro --> ZIP or DMG.
Along with an appropriate description this might still be OK if you have one macro file and one script/bin file.
But download the LB Action provided as an example in the first post: As a KM macro, how will you instruct the user to put that in place? OK, a centralized directory. But then comes the next macro that overrides a folder there…
So, first you folks convince me that macros are so dangerous that they should be installed disabled by default, and I come around to agreeing that yes, macros should be to allow the user a chance to inspect them, and now you all want a bundle format so that the user can easily install pre-compiled executables without ever having to look at them…?
Aren’t there serious security implications if I go around installing pre-compiled scripts without giving the user any sensible opportunity to even see the code?
I can see the problem you are wanting to resolve, but I’m not sure that it’s a good idea to install stuff for the user without them seeing anything.
Similarly, including extra things in an opaque package when you export macros sounds problematic from a privacy point of view.
Certainly it would be easy enough to have a standard folder in the Keyboard Maestro Preferences folder (which you can always open with the Help menu), and then can access with
Isn’t that what happens with any program you install?
Usually I code sign my compiled scripts (unless I forget it). While this doesn’t prevent them from executing bogus code, at least it makes sure that the file hasn’t been altered afterwards.
But besides that, yes, the user has to trust that the compiled script corresponds to the source code as posted on the forum.
But security is also an issue with uncompiled scripts: If you download a macro that contains 150 lines of AppleScriptObjC, and you want to be sure it doesn’t do anything fatal you should run it stepwise thru Script Debugger in debug mode. How many people will actually do that?
I know, this comparison is a bit unfair, since with uncompiled scripts the user at least does have a chance to check the code. But if the user doesn’t do it —or simply doesn’t know how to do it— the end result is the same.
Similarly, including extra things in an opaque package when you export macros sounds problematic from a privacy point of view.
You can always check the package contents before uploading it. But why should the macro include stuff I haven’t told him to include? And, is a bundle/package really more opaque than a kmmacros XML file with encoded rich-text streams?
So, first you folks convince me that macros are so dangerous that they should be installed disabled by default […]
Like with any app or script, don't run unless you obtained it from a trusted source.
KM should continue to disable macros when imported. Thus, whether malicious code is in plain sight or hidden in a bundle, it would not immediately affect the user.
In the case of a bundle, or some form of external resource, when it is imported, KM should:
Notify the user of the bundle/resource, providing its name and author if possible
Ask the user if should be imported
Ask the user if they would like to view the resource
Ask the user if they want to delete the resource
Whether the macro contains a script as text in an Action, or as an external file, the user should open the script for viewing/editing in the appropriate editor, and carefully review the script.
Please don't store the scripts or resources in the ~/Library folder. They are much harder to find and search there, and are not easily found by a macOS Spotlight search. I'd suggest: ~/Documents/Keyboard Maestro/Scripts
But having an official folder for KM scripts would be great. At the very minimum we could provide scripts and other resources to be stored there by macros we are uploading.
Just want to say that I would personally hate it being put into ~/Documents as I use Documents for other things and I want it to be clean of app configurations and scripts. Other than that I do think that sharing bundles will open KM to some new and more powerful macros we can share.
First, a standard supporting-files directory (or set of directories) in application support would be very, very, nice. I think it should be added, and I think Custom HTML Prompts should have a standard go-to root directory for supporting images, scripts, etc.
Secondly, if macros were not in a giant file but were instead discrete items in the Finder, that would give us some potential to group stuff together without opaque package files. So I could share a folder (zipped, presumably) with a macro file, some images, and a script all in the folder. If that sits in ~/Library/Application Support/Keyboard Maestro/Macros/ , its contained macro would get pulled in by KM and made available for use, and could access its supporting elements that are in the same path of it.
Thirdly - assuming less monolithic macros, synchronization would be simpler, as would managing, sharing, and versioning specific macros (e.g. via git). I could synchronize my whole git repo of shared macros into, say, ~/Library/Application Support/Keyboard Maestro/Macros/My Fancy Macros/Macro 1… and they’d be available for use.
As for how this works programmatically? Well, I’m just creating problems for you. But potentially KM could “compile” its XML file at launch, rebuilding it occasionally as it goes while things are edited.
I understand what you are saying, but please consider this: custom scripts for KM are just like documents for some of your other apps.
The issue is when you want to search for these scripts, the standard macOS Spotlight will NOT find them in the ~/Library folder. Also, this folder is not even visible in the Finder unless you hold down the option key and select it.
Many other apps put their user created files in the ~/Documents/AppName folder.
Of course, one other option is for KM to create/maintain a Spotlight index file. That would be OK with me, but I'm thinking that it is more work for the KM developer, and also more fragile, subject to issues.
But you are always overcomplicating things …
