When I write macros, sometimes I need to save information, like the position of a Custom HTML Prompt for example. Normally I store them in a global variable prefixed with "DND__", because this is a sort-of convention that our dearly-missed brother, @JMichaelTX, was an advocate of.
But of course the size of these variables starts to add up, especially if I store other information in them. So for some of them, I was thinking I could save them to disk.
So what I'd like to know is, where's a good place to save this information? Should I use Library/Application Support, and a folder like "com.dagware.km" ("dagware.com" is my personal domain name)?
Should I store it in a subfolder of KM's Application Support folder? Somewhere else?
I don't really like the idea of using a subfolder of Documents, because I don't want to "pollute" that namespace.
I cannot comment on "best practice", but speaking as just one Mac user, I would be happiest to see the data stored in a unique subfolder of ~/Library/Application Support/ e.g. in "Keyboard Maestro – Dagware macros" or "Keyboard Maestro – Dan Thomas's macros".
Naming the folder like that is maybe too idiosyncratic, but it's transparent. I see folders named along the lines of "com.dagware..." in the Application Support folder, but they are in the minority. I would expect that sort of cryptic (and IMO, rather unwieldy) naming only in the Preferences folders, where they are the standard.
I would say that that is more an issue of polluting the user's space. However, so many applications dump stuff in that folder that those of us who care about keeping track of our files have to store them elsewhere anyway – even if it's only in a subfolder of "Documents" – so keeping that space clean is a long-lost cause. Nevertheless I see no point in contributing to the chaos there!
I know it's not always possible, but if it is, I have the macro ask the user where to store the data. I then keep that path in a global. When the macro launches, it checks for the existence of the global; if it doesn't exist, the macro assumes it's the first run and asks where to store its data. If it exists but the data can't be found at that path, the macro asks the user to find the new location.
If I have to store something without asking the user's permission, I tend to put it a blatantly-named folder in Documents, because Apple hides the user's Library folder from them by default and I want to be transparent about the fact that I'm storing something on their drive. (I think I've only ever done this once, though. I ask in almost all cases.)
That's actually a good idea. I wonder if, once the user answers the question for the first time, you could use that location for data for other macros, without the need to ask again?
I think you could, if you worded it generically: "This macro, and others that I may distribute, need to store some local data on your Mac. Please select the folder where you'd like that data stored." As a user, I'd be fine with something like that.
In ownCloud (similar to Dropbox) I have a folder called AppData/Keyboard Maestro. In that folder goes my sync file and a km_data folder. That's where I store any data written by KM.
One example is that I have a macro that creates an 'import template' for Mylio photos, and increments the 'import number' by one each time it's run. I store a file that simply has the last number used. That way it works across devices.
Another nice side effect of storing files in ownCloud is that they can be accessed via iOS and used in Shortcuts if need be. They can also be read by KM on other machines (again, if need be).
The idea occurred to me too, but I did not suggest it for the following reasons:
Implementing that would be more work for you.
It's another factor that might be involved in something going wrong.
Most users will want to stick with the default, especially if the default seems sensible.
If users could change the location of the folder, that would be another variable when troubleshooting.
In other words: since you are giving such serious thought to the matter, I think it unlikely that you will make a decision that will inconvenience any users to such a degree that adding any complexity to such a non-issue would be justified.
I was minded that that folder should be left alone, since KM has no responsibility for third-party data in that folder. For example: what if a new version of KM were to store data in a new location, and migrate and translate to a new format only the data it recognised? However unlikely such an incident might be, invading KM's space without co-operation seems imprudent.
However, if you could persuade Peter to recognise an official subfolder for such purposes, that might be the ideal solution. I imagine that would be a policy decision based on how many other macro developers would use such a folder.
So that would be something like:
~/Application Support/Keyboard Maestro/User Macro Data
– with data connected to your macros being saved somewhere such as:
I thought about this also. I think, if anything, I'd like to see a "more official" way of handling this - something built into KM. But honestly, Peter has surely got better things to do than this.
Unless you are talking about excessive or non-text data, variables seems a fine solution - variables are stored in an SQLite database so it really should not be an issue unless you are talking megabytes.
Other than that, it's hard to say what a good place it. ~/Library/Application Support/Keyboard Maestro would seem reasonable, but who knows if Apple will allow scripts to write there. Same for ~/Documents/something (which scripts probably cant write to unless permission is granted).
As noted that folder is never going to be clean and yes it rubs me the wrong way as well.
My simplistic work around is to give MY folders in the Documents folder a Finder tag and clicking on that tag in the Finder's sidebar gives me a pristine clean list to work from. I then don't look in and bother myself with the dumping ground the actual Documents folder has become.
The problem of squatters in "Documents" is, of course, not new, and for as long as I can now remember, I have used a subfolder in "Documents" for my own files. Then (even without involving KM), it is easy to set things up so that that subfolder is just as easy to access as the "Documents" folder.
There's a "Documents" folder? I always assumed that was an autocorrect error from people typing in "Desktop"...
More seriously -- @DanThomas, what's the intended behaviour of these variables for people with more than one Mac? To sync, or not to sync, that is the question... You may even want to use two locations, ~/Library/Application Support/ for machine-local variables and ~/Documents/Dan's_Place for user-global (iCloud-syncable) variables.
Yes and especially so if you have done so ongoingly. Changing pathways by adding a subfolder might necessitate fixing some less than robust references.
I have many Variables that are in sync between my Machines and even my VM‘s …
They are all stored in a Subfolder called Synced Global KMVAR‘s inside of my Pseudo Package File in my Dropbox where I store all my KM related Data.
Some of them are quite heavy because of Base64 encoded Image Data, some are large JSON Files (up to 6 nodes deep nested.
Even if KM could probably handle some of them - I have them stored there for easy access with any Text Editor if I need to have access.
Whenever the contents of these Variables are needed I recreate the Variables on the fly just before the Macros which need access to them are called.
I hope this gives you an idea of how I handle some of my Variables.
For the vast majority of them, they don't need to be synced. They're usually just variables to remember a window's size and position, and a few options that might be turned on or off.
As much as "do they need to be", consider "must not be" -- for example, window size and position may not be a good candidate for syncing between a 32" display desktop and a 13" Air. So, unless you're also including machine IDs or similar, you wouldn't want a record of that stored in ~/Documents.
I don't know about storing it in Documents or not, but as long as it operates the same way most applications do, that's fine by me.
In other words, I don't know of many, or even any applications that save the window position differently depending on which monitor you launch it on. As long as the window doesn't disappear off the screen, that's good enough for me.
It's not as though I'm writing professional applications here. You could spend forever trying to cover all bases - bases that most applications don't bother with.
Say I've got KM installed, and syncing, on two machines -- one's a desktop with a 32" display, the other is a 13" Air. The machines sync ~/Documents via iCloud.
If the settings are stored in a file/folder in ~/Documents then any change I make for the 32" display will be applied on the Air, which could be a problem. I wouldn't be able to set options on one machine and not the other, and so on.
That may, or may not, be an issue. But it might be something to consider when deciding where you want to save the data.
