MACRO: MacroBackerUpper—it's like Time Machine for your macro library

Every time I try to run MBU I get the following... A backup folder is created in $HOME with my KM macros but it hangs at the "administration" dialog (and obviously doesn't move the folder to its designated destination). When I first ran it I did have some identically named macros, but I've consolidated those now. I've also tried to reinstall / re-run having first deleted all the 'rg_' variables.

2023-11-12 10:42:23 Execute a Shell Script failed with script error: Runtime error near line 16: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 17: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 18: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 19: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 20: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 21: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 22: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 23: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 24: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 25: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 26: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 27: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 28: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 29: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 30: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 31: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 32: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 33: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 34: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 35: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 36: UNIQUE constraint failed: groups_new.Group_ID (19)
Runtime error near line 37: UNIQUE constraint failed: groups_new.Group_ID (19) in macro “•main | MacroBackerUpper” (while executing Create/update the database).
2023-11-12 10:42:23 Execute macro “subs | 94 Control export window” from trigger Execute Macro
2023-11-12 10:42:28 Action 15365556 failed: Split Path failed because with path %Variable%rg_MBU_tNewestBackup%
2023-11-12 10:42:28 Split Path failed because with path %Variable%rg_MBU_tNewestBackup%. Macro “•main | MacroBackerUpper” cancelled (while executing Split Path “%Variable%rg_MBU_tNewestBackup%”).

I just added some troubleshooting steps to the help page, but this problem occurs when MBU didn't finish properly, leaving the database in a weird state. The easy fix:

  1. Delete the latest backup.
  2. Delete the zDatabase.db file in the zSupport Files - Do not delete folder at the top level of the backups folder.
  3. Re-run MBU.

You shouldn't need to delete either of the global variables, though, unless the path is wrong in rg_MBU_BackupLoc.

Please let me know if this works. I'm also working on some logic to recover/notify if such an issue is found on launch.

-rob.

Hi Rob, thanks for your help! Sadly that didn't address it. I deleted the 'KM Backups' folder entirely (including the zSupport Files folder), and deleted the rg_ variables for good measure, then re-ran accepting the defaults (i.e. 'KM Backups' into ~/Documents). Same issue, hanging on the 'variable-setting administrativia' popup.

Can you look in the log and let me know if there's an error message? Alternatively, if the macro is still running (menu bar icon spinning), select Start Debugging from the menu bar icon and let me know which command it's working on.

thanks;
-rob.

Now I'm getting these messages:

2023-11-12 15:32:34 Execute macro “•main | MacroBackerUpper” from trigger Editor
2023-11-12 15:32:38 GetProcessForPID(2359) failed with -600 for <KBMRunningApplication “Keyboard Maestro” <0x60000002c8f0> pid=2359 bid=com.stairways.keyboardmaestro.editor Abht>, using cached version
2023-11-12 15:32:38 Action 15365916 failed: Manipulate Window could not find any matching windows
2023-11-12 15:32:39 Execute macro “subs | 94 Control export window” from trigger Execute Macro
2023-11-12 15:32:44 Action 15365556 failed: Split Path failed because with path %Variable%rg_MBU_tNewestBackup%
2023-11-12 15:32:44 Split Path failed because with path %Variable%rg_MBU_tNewestBackup%. Macro “•main | MacroBackerUpper” cancelled (while executing Split Path “%Variable%rg_MBU_tNewestBackup%”).

It's hanging on subs | 94. I also switched off and on KM's screen recording privileges in System Settings.

Ugh, this appears to be the -600 AppleScript error that's not related to the macro at all. I have some code in there that's supposed to help, but doesn't seem to be in your case.

As a test, select any macro, then use File > Export > Export as Folder. When that's done (you can delete the export), try the macro again.

-rob.

Sorry, no joy :frowning:

2023-11-12 16:01:51 Engine Starting 11.0.1
2023-11-12 16:01:59 Execute macro “•main | MacroBackerUpper” from trigger Editor
2023-11-12 16:02:02 GetProcessForPID(2694) failed with -600 for <KBMRunningApplication “Keyboard Maestro” <0x6000004ac420> pid=2694 bid=com.stairways.keyboardmaestro.editor Abht>, using cached version
2023-11-12 16:02:02 Action 15365916 failed: Manipulate Window could not find any matching windows
2023-11-12 16:02:03 Variable “KMVAR_instanceTheData” excluded from environment to ensure the environment is not too large
2023-11-12 16:02:04 Execute macro “subs | 94 Control export window” from trigger Execute Macro
2023-11-12 16:02:09 Action 15365556 failed: Split Path failed because with path %Variable%rg_MBU_tNewestBackup%
2023-11-12 16:02:09 Split Path failed because with path %Variable%rg_MBU_tNewestBackup%. Macro “•main | MacroBackerUpper” cancelled (while executing Split Path “%Variable%rg_MBU_tNewestBackup%”).

While I work on some more AppleScript code, does it work if you quit and relaunch the Keyboard Maestro engine?

-rob.

Sorry I left in the 'Engine Starting' comment above but didn't mention that that's exactly what I did. Also -- probably not relevant as I can see you're internationalising using the 'one true date format' (ISO 8601) but I'm in a GMT timezone.

I don't think it's time related at all. I'm going to release 1.3 shortly with some AppleScript in it from Chris that should hopefully avoid the -600 issue. When it comes out, please let me know if it works for you.

-rob.

1 Like

Version 1.3 is out now with some nice improvements, some troubleshooting help, and a fix for a bug that would crop up in rare situations. If you're on 1.2, you should be able to run the update checker (upd | Update check macro) to download version 1.3. Or it's linked above in the first message in this post.

Here are the full release notes:

  • If MBU ever doesn't finish a run, it can leave the database in a weird state. A new routine detects that state, which indicates the last backup had issues, and then cleans things up: It deletes the last backup, the __latest alias, and the database. The database is then rebuilt before the next backup begins.
  • When the final summary report is set to show 'Only if macro manually triggered,' it will now show if you trigger via palettes (in addition to hot keys or the Editor window).
  • Added some code to some of the AppleScripts to try to avoid a not-macro-caused -600 error that a few users are getting on random occasions.
  • Fixed a bug that, under very rare conditions, could cause a hard link to not be created between the newest and previous versions of a macro.
  • Reorganzied the macros a bit, moving the update checker into 'main' so it's more findable, and moving the Settings Manager out of 'main' as it's not intended to be run on its own.

Thanks to cdthomer for the bits of AppleScript related to the -600 errors, and for some ideas to reduce console logging and handle the TriggerValue in a more-elegant manner.

-rob.

4 Likes

I did a clean install of 1.3

2023-11-13 07:09:05 Execute macro “•main | MacroBackerUpper” from trigger Editor
2023-11-13 07:09:23 GetProcessForPID(2694) failed with -600 for <KBMRunningApplication “Keyboard Maestro” <0x6000004ac420> pid=2694 bid=com.stairways.keyboardmaestro.editor Abht>, using cached version
2023-11-13 07:09:23 Action 15372476 failed: Manipulate Window could not find any matching windows
2023-11-13 07:09:24 Action 15372489 failed: Execute a Shell Script failed with script error: text-script: line 1: syntax error near unexpected token `fi'
text-script: line 1: `if -test /tmp/MBU_rg*; rm /tmp/MBU_rg*; fi'
2023-11-13 07:09:24 Variable “KMVAR_local_TheData” excluded from environment to ensure the environment is not too large
2023-11-13 07:09:25 Execute macro “subs | 94 Control export window” from trigger Execute Macro
2023-11-13 07:09:30 Action 15372099 failed: Split Path failed because with path %Variable%rg_MBU_tNewestBackup%
2023-11-13 07:09:30 Split Path failed because with path %Variable%rg_MBU_tNewestBackup%. Macro “•main | MacroBackerUpper” cancelled (while executing Split Path “%Variable%rg_MBU_tNewestBackup%”).

But did the backup run? Nothing there should have blocked execution of the macro—I just did a clean install test and had some similar errors, but the backup ran.

I'm going to DM you a test build to see if we can figure out what's going on.

-rob.

Version 1.4 is out now; here are the full release notes:

  • Found and fixed a bad typo regression (m instead of rm!) that I made in 1.3 in one of the hard link commands.
  • Added window check for KM's Preferences window, so the macro only closes it if it's open.
  • Formatted time display in summary dialog to tenths instead of thousandths.
  • Further simplified and improved handling of moved and/or renamed macros.
  • Removed routine that tried to position the export dialog window, as it was causing all sorts of issues.
  • Positioned a screenshot (shown during setup) where it should have been positioned according to its code, but wasn't.
  • Did some more work to try to stop the dreaded -600 errors seen by some users in some AppleScripts. I don't think it will work, but these shouldn't be errors that prevent the macro from running.
  • Further cleaned up some shell scripts to prevent spurious log entries.
  • I may have found a way around the "Type 1" errors being logged as the result of diff calculations. Please let me know if you see any, and what the associated action number is in the log for the error.

I'm hoping the change to remove the export window manipulation fixes the issue that harlequinade and others may have been having with even getting the macro to run. Please let me know if it works for you.

-rob.

3 Likes

Hi Rob, sorry for not responding before. Similar to earlier builds, v1.4 creates the backup in $HOME (i.e. in a folder called "2023-11-14_06•51•21"). But it still hangs at "Doing some variable-setting administrivia" and throws a Macro Cancelled error ("Split Path failed...") per the log.

2023-11-14 06:51:16 Execute macro “•main | MacroBackerUpper” from trigger Editor
2023-11-14 06:51:20 GetProcessForPID(11398) failed with -600 for <KBMRunningApplication “Keyboard Maestro” <0x6000004d00b0> pid=11398 bid=com.stairways.keyboardmaestro.editor Abht>, using cached version
2023-11-14 06:51:21 Variable “KMVAR_instanceTheData” excluded from environment to ensure the environment is not too large
2023-11-14 06:51:26 Action 15377196 failed: Split Path failed because with path %Variable%rg_MBU_tNewestBackup%
2023-11-14 06:51:26 Split Path failed because with path %Variable%rg_MBU_tNewestBackup%. Macro “•main | MacroBackerUpper” cancelled (while executing Split Path “%Variable%rg_MBU_tNewestBackup%”).

I had deleted v1.3 and installed v1.4 fresh; I had also deleted the "zSupport Files - Do not delete" folder. Sorry, I know you are putting a lot of work into this and I can't think what is in my environment that is preventing MBU from fully working.

Let's take this to direct messages to figure out what's going on; I'll send you a note shortly, and sorry you're having so many troubles with it! :frowning:

-rob.

Following up on the issues harlequinade is having with the macro: After sending him a number of test builds, I couldn't figure out the problem at all. Thanks to technology, though, we were able to screen share between the UK and USA, and discovered ... weirdness.

For some reason, his two Macs behave inconsistently (and differently from any other Mac I've seen) with the Shift-Command-G shortcut I use to paste the path for saving the backup. For some folders, it works perfectly—he can make his backup in a nested sub-folder on his Desktop, and it works fine. But it won't work in his Downloads folder. Or his Documents folder.

More confusingly, the Shift-Command-G shortcut works fine in Finder. And works fine in other apps. It's just inconsistent in Keyboard Maestro. But that dialog is a macOS dialog; I'm pretty sure Keyboard Maestro isn't involved at all once it's onscreen.

If you have a similar issue (macro doesn't successfully run a backup, then ends abruptly), I'd love to hear from you. Unfortunately, I have to interact with that dialog box to set the save path, and I'm not sure I can do it any other way short of mimicking selections and arrow keys in the file list section of the window. And that seems fraught with danger. So for now, at least, he can only back up in certain locations :(.

-rob.

4 Likes

A bit more on this issue: If you have your backups set to go into Documents, and Documents is not in your Finder sidebar, the macro will not be able to locate the target folder. Despite entering a full path to a folder in the Documents folder, the Export as Folders dialog won't select that folder.

This is not a Keyboard Maestro bug, as you can see the same thing in Preview in the Open dialog. Try to navigate to a folder in your Documents folder that contains some images via the / or Shift-Cmd-G path entry shortcuts, and it won't work unless Documents is in your sidebar.

I suspect Apple has some special setting for the Documents folder (and maybe other folders at the top level of the Users folder?). Unfortunately, I know of no way to get a list of folders in the sidebar, so this is an impossible condition to test for in the macro.

As a workaround, MBU 2.0, which will be out very soon, will verify the backup when it completes. If it's not where it's supposed to be, then future backups will require you to manually select the backup folder each time, until you add Documents to your sidebar. (After doing so, you can use the Settings Manager to disable the manual mode.)

I wish I had a better solution for this, but that's all I could come up with after some serious brainstorming.

-rob.

Could you export to \tmp\ and then move the files with KM afterwards?

1 Like

Ah, that's a good thought! I'll play around with it tomorrow, thanks!

-rob.