MACRO: MacroBackerUpper—it's like Time Machine for your macro library

Is the macro still running? Check in the KM menu bar icon. If it is, choose Start Debugging and then grab a screenshot and send it to me as a direct message—I'm trying to keep debugging out of the main thread :).

If it's not still running, send me a DM and I'll explain how to find a log entry which will help me figure out what happened.

-rob.

MacroBackerUpper 2.0 is out! Yes, a major version bump a few weeks after release. I had to do it this way, though, as there are changes in the database, as well as a new settings database. With those changes, downgrading to 1.4 from 2.0 will require a bit of work (explained below). But 2.0 is very well tested, and has tons of new features and optimizations.

Before upgrading, I recommend running 1.4 again, so you have a backup with a change report for your most-recent changes. Due to the structural changes, the first 2.0 backup will be a full (no hard links) backup, and there won't be any comparisons with the previous backup.

What's new

  • Finder labels can be turned off! This was asked for by a few people, and now it's possible. Prior backups won't change, but all future ones will.
  • You can change the countdown timer delay from 3 to whatever you like—anything from 1 to whatever, as long as it's an integer.
  • You can control the timing feature in the report via settings, instead of editing a bit of the main macro.
  • Groups that were enabled or disabled since the last backup are tracked, and the macros within them removed from further analysis, as they will always report being modified.

I fixed and improved a slew of other things, including replacing the inefficient (for me to keep updated!) Progress action calls with a task-based progress meter of sorts. You can see it in action in this demo macro.

I also added a lot more safety checking to the macro, to make sure that things are in the right spot, with the right names. In addition, all backups are first written to /tmp and then moved into your backup folder when done. This should completely avoid an issue some users were having with the right folder not being selected. Thanks, JimmyHartington!

Click to see all the other small fixes and changes
  • Settings are now stored in an external database, which makes it much easier to have more of them, and much easier to manage them. There's also now just one global variable for the macro, the one that holds the location of the backups.

  • Replaced the impossible-to-maintain progress bars with an onscreen task list. Each task gets checked off when it's completed.

  • Fixed a typo that prevented the backup count limit from ever being enforced. Whoops.

  • Fixed Keyboard Maestro window not returning to its prior location after a backup run, I hope.

  • Now testing for backup and support folder existence on macro launch, to hopefully prevent issues before they get worse.

  • Added a loop to confirm the latest backup folder has been created and properly named before starting the backup (well, yeah, that seems logical…why didn't you do that before?!)

  • Backups are now made first to the /tmp folder, which should work for absolutely everyone, then moved using the Move action. This should completely avoid the trouble some users were having when Documents wasn't in their Finder sidebar.

  • In case the above doesn't work for everyone, added a backup check to make sure the folder is where it should be, and handle things if it's not.

  • Also added a check to make sure the global backup folder location variable exists right after it was supposedly created, just in case something really weird happens.

  • Made the Cancel button in the "update check" window actually cancel.

  • Found a better way to make sure the diff calculations don't throw Type 1 errors into the logs: The || true construct, which forces any command to exit with a status of 0, not 1.

  • Reworked the Settings Manager for more consistency and better readability.

  • Fixed the Settings Manager option for showing timing in report to default to no, which is what it does on initial setup.

  • Substantially reworked the logic and flow of the main macro, to make sure things are in place before trying a backup.

  • Fancied up the initial delay splash screen, just a bit.

  • An upgrade script runs on first run, and handles upgrading users of pre 2.x versions of MacroBackerUpper.

Upgrading from 1.x

The macro will determine if you've run 1.x before, and on initial launch of this version, it will perform the required database changes. You'll have one more chance to back out and run the old version first, in case you want a full "what's changed" report prior to upgrading.

If you want to be 100% certain you can downgrade to 1.4 if you want to, please follow the instructions in the disclosure area below. Note that I don't think this is necessary, as the upgrade process works and I've tested it literally hundreds of times. But if you want to make sure you can downgrade, just click the triangle.

Prepare in order to downgrade if desired

Do all of this before running version 2.0:

  1. Use a Finder label or tag to label the latest current backup with version 1.4, just so you can easily spot it.
  2. Copy the rg_MBU_OtherSettings global variable to some temp global, just in case you want to put it back (it's deleted after the upgrade). Alternatively, copy its values somewhere safe so you can recreate it if needed.
  3. In the zSupport Files - Do not delete folder in your backups folder, duplicate the zDatabase.db file and name it zDatabase Pre Upgrade.db, or something like that.

You can now run 2.0, and be able to downgrade if you want/need to. To do that, here's what you'd do:

  1. Delete the backup made under 2.0.
  2. Delete the zSettings.db file you'll find in the zSupport Files - Do not delete folder in your backups folder.
  3. Delete the zDatabase.db file, then duplicate the backup you made, and rename the duplicate to zDatabase.db.
  4. Recreate the rg_MBU_OtherSettings global variable.

You can now run 1.4 again as if nothing happened. If you want to try 2.0 again, I recommend just deleting the one you have installed and downloading a fresh copy.

I have done these things so many times in the course of testing that I'm confident they'll work if you need them. But I'm just as confident you won't need them, as 2.0 runs fine.

If you have any issues with this version, please let me know!

-rob.

4 Likes

This is very impressive.
Love the new progress window :slight_smile:

3 Likes

v2.0 works brilliantly and a great solution from @JimmyHartington and @griffman to solve my ~/Documents issue. Thank you! The progress window is a significant enhancements as well.

3 Likes

Amazing work. The progress window is a terrific innovation.

Thanks so much for your effort, @griffman. This macro should be one of the first macros all Keyboard Maestro users download, install, and use!

1 Like

Excellent utility @griffman! MacroBackerUpper 2.0 should be added to the best macro list!

2 Likes

I'm working on an issue with 2.0 that seems related to starting the macro from outside the KM Editor. For now, please run it only inside the editor. I'm not sure what's going on just yet, but I'm diving in and will have a fix out ASAP.

-rob.

I just installed v2 after doing a final v1.4 backup and ran into problems.

First I thought I'd look at the settings to check how many backups I'd set to save. It said 50 (correct) and as I didn't want to change that I cancelled out of the setting and then exited from the settings panel. The backup process started and I got this dialog up:

image

With no obvious way to actually enter a number I cancelled the whole process.

Next, I ran v2 again - this time I didn't explore the settings and just let the backup go. It got this far:

image

and I received an error notification that said:

image

which referred to this action in the v2 macro:

Since the backup macro failed I haven't tried running it again.

Fix please!

Weird. I'll try to replicate that here and see if I can.

Yea, the problem is by that point, the database was messed up. Can you please run it once more and let me know if you see a "Database unusable" dialog? If you do, proceed with the backup and let me know what happens.

thanks;
-rob.

I use 9 different machines.
Will it be a problem backing up on the different machines?
I ran V2 on my main cpu this morning successfully.
I just ran V2 on another cpu and it stopped on the Analysis:Enabled and disabled groups.

This is not a biggie, I'll just run it daily in the am on my primary machine.
Just wanted to give you feedback.

Thank you for all the incredible work you've done on this macro!

Cheers.

Troy:

It only backs up on the Mac it was run on, and it should run fine on any of the Macs—I test it on two here. I assume you're backing up to folders directly on each Mac?

-rob.

Yes I got that so I let it continue - which it did with no further problems. Ran v2 again and it went smoothly.

Nice!

Glad to hear it—I'm working on your issue and another, and will have a minor update out once I figure out what's going on.

700+ tests here on two Macs, zero issues ... release, and issues :).

-rob.

Backing up to a dropbox folder common to all cpu's.

My KM set is common to all 9 cpu's as well.

I do not recommend doing that. I haven 't tested it with any sort of cloud-based backup, and if you're backing up multiple Macs to one folder, that is most definitely going to cause an issue of some sort.

MBU is designed as a local backup tool for the macros on one Mac. Anything beyond that is untested and probably non functional :).

-rob.

1 Like

!!!!!

I read the warnings along the way of installing and using MBU, and went ahead and set the MBU destination to a dropbox folder anyway; however, I only run backups from one machine at home... is that maybe ok? :thinking::sweat_smile:

Hey, Rob.

I did a final backup on 1.4, ran 2.0 and it worked fine. Then I changed a random macro and re-ran MBU to look at the report. I got an error, ran again and it said the database was unusable, ran it again to rebuild, no luck.

Here's the full error:

failed: Execute a Shell Script failed with script error: Error: near line 2: in prepare, no such table: macros_new (1)

It should probably be OK, but I know that we have had issues with our app Name Mangler due to Finder getting out of sync with Dropbox when we're modifying a bunch of files—basically Dropbox falls behind Finder, and we can get caught in a loop where it looks like there's a filename conflict but there's not.

But with this, as long as the backup ran, that's pretty much it for file modifications, so it should be OK.

-rob.

1 Like

Can you message me the full log from the start of that run? I have no idea why it's now failing for some folks after a successful upgrade.

-rob.