Any interest in a personal "useful forum macros" collection tool?

There is so much good stuff that gets posted here, and I've tried various ways of keeping up with it: The forums' bookmarking tool, Safari's bookmarks, notes files. All of them fail, primarily because it's still too hard to find what I want. And it's work to remember to add the items to another app, or to keep my bookmarks sorted, etc.

This morning I had the thought that a "best macros collector" macro could be just what I wanted to make it easier to save, find, and use all the macros posted here. So I whipped up a rough proof of concept, and so far, I like how it works. Here's the macro group, which currently only works in Safari.

__Best Macros Collection Macros.kmmacros (40 KB)

How it works: Hover over any macro link—like the one above—and press Control-C. The macro will save the download link, macro title, and page URL, then present a pop-up categorizer:

image

Select an existing category or add a new one, then press Enter, and the macro's details are saved to a SQLITE3 database. Once you have at least one item in your collection, while in Safari press Shift-Control-C to open the collection browser:

Select a macro in the list, and you get its details dialog:

Open Topic will open the thread with the macro, usually on or close to the post where the macro was poste; Download will download the macro.

This is a very rough concept—the final version wouldn't use dialogs, but HTML prompts with tables, like my Recent Macro Browser Plus macro, so you could sort and act on the entire database of entries at once. And you could search the database, of course, using the description or title, etc. And there'd be ways to delete, edit, etc. the database entries, and a lot more.

But before I get too far into it, is there general interest in a tool like this? If so, I'll develop with sharing in mind. If not, I'll just do what I need to make it useful to myself, which will require a fair bit less effort—I don't mind doing the work, but if there's no interest, I won't bother :).

-rob.

4 Likes

Hello Rob :wave:

I‘d be interested to see what this may be look like and would also love to use such a tool … even though it currently doesn’t exist I would say that this would be possibly a nearly dayly used one with the by far most usage count of all macros used on that particular day.

Greetings from Germany :de:

Tobias

1 Like

Very interested!

Beyond Chrome, Firefox, and Safari, what other browsers should it (if possible) support?

-rob.

Hello Rob (@griffman) :wave:

Maybe these ones ?

  • Arc,
  • Brave,
  • (MS-)Edge,
  • Orion &
  • Vivaldi

Greetings from Germany :de:

Tobias

Hi @griffman
This looks really good.

I'm not familiar with SQLITE3 databases. Can the location of the database be changed to a location chosen by the user, such as a location on Dropbox?

1 Like

Hello Aurio (@Zabobon) :wave:

Yes, you can even place them in a Directory inside Dropbox and share them with other Macs. Or even VM setups.

Greetings from Germany :de:

Tobias

1 Like

Hello Rob (@griffman)

Downloaded your first draft Macros and there is something you should consider…

It’s Localization… here is my current Contextual Menu on the KM Forum Page as an example:

As you can see there’s a Menu starting with the Letter C that in my case would trigger my Clip to Quiver Extension.

I thought that sharing this - could be very use- and helpful to you on your development plans.

For now I’m going to fix this Issue on my side.

EDIT:

Another thing to mention is to verify you’re collecting a link to an uploaded kmmacros file or any Zipped Content that may contain such a file to first check if the Link is valid:

^(https://forum.keyboardmaestro.com/uploads/short-url/*.(kmmacros|zip|kmmacros\.zip|scpt))$

I’m really bad when it comes to RegEx but this should give you at least the idea what I’m suggesting here for the macro to continue after your forum url check (if my attempt is not correct). And if this Match doesn’t exist to remind the user to place the Mouse correctly and start again collecting data.

Greetings from Germany :de:

Tobias

Different solutions appeal to different mindsets, so although this collection tool does not appeal to me, I do not mean to dismiss it. In case it's of any interest though, here's why I would not use such a tool: I believe that the task is just one instance of a more general problem that already has adequate solutions.

What makes it hard to find what you want? I would imagine it must be either (a) lack of adequate descriptors or (b) inadequate search features. You say you have tried using notes files, but case (b) can be handled just by using a plain text file (and of course any PIM that uses them). Case (a) would apply to any system in which you haven't properly described the macro, but with a plain text file you have instant access to editing at the time of creation and at any time thereafter (manually or via a macro), and, failing all else, you can run your eye over the whole page!

Could the problem perhaps be that your attention is already divided between "The forums' bookmarking tool, Safari's bookmarks, notes files"? If so, adding another tool will just give you more pause to think when saving data ("Ah, now it's this kind of data, so I need to use, umm, this solution" versus "I can add as text to a note, along with most other data").

I am making a case for using fewer, more generalised solutions for as many tasks as possible rather than multiplying solutions.

All you need to save is the page title and the URL. Since there are many ways to do that, from any browser, why extend the problem space and why reinvent the wheel?

There is probably no need to download a macro other than as insurance against this forum suddenly disappearing, but if you do so, I admit that some extra complexity enters a system based on notes: you will want to way to uniquely identify that macro from a reference in the note. However, that is an implementation detail since that could be handled in many ways.

How is that superior to using plain text? You can access text files with any tool you have to hand, they don't take up much disk space, they are transportable to any other system or user, and they aren't as vulnerable to total data loss as a binary database. Although the SWLite site assures us that "SQLite databases are remarkably rebust" (sic! I hope that wasn't cause by database corruption :wink: ), it goes on to mention possible dangers before documenting its recovery API. [1] That's assuming that no backup is available, of course. To try to fix corrupt text files, you just need a text editor.


  1. Recovering Data From A Corrupt SQLite Database ↩︎

Blah blah blah :slight_smile: I suppose the key question is: do you want to use KM not only as a library of your macros but also as a database of macros that you might want to use one day? I can see that for many it would make sense to keep macros and information about available macros in one place. To me, that's extending KM's role, in a proprietary way, to handle a different kind of (meta)data.

However despite my case arguing against such a system, I can well imagine the appeal in both practical and conceptual terms to some users, so I do not mean to dismiss it! I hope that a counterargument gives some food for thought, because (as is touched upon by the opening of the original post) deciding on how to handle data can be tricky, and no solution will suit everyone.

OK, maybe not a hammer.

Makes a note.

1 Like

Hello Kevin (@kevinb):wave:

I think having a Macro like this is gold despite on how it is storing the data.

Here is the reason: let’s say I am surfing the forum and I have 120 macros in total I want to bookmark and or like but the forum doesn’t allow me to do so - maybe I have permission to bookmark 30 and like 20 and after that I’ll have to wait 24 hours to come back for another 30 Bookmarks and 20 likes … and so on …

Maybe you’ll get the idea … despite I don’t have the time and energy to come back after 24 hours for this task I could use this Macro to open the Macro‘s Post and can like when ever I have the time to do it .. same thing for a bookmark - but here’s the next bummer you’re allowed to have 2000 bookmarks in total …

And if you’ve no time to reference everything outside your browser in your preferred Applications this bookmarks will grow in count very fast … and I am facing this situation currently as I have nearly 2000 bookmarks and no time for referencing this information in text anywhere on my Mac.

If I had a macro like this I wouldn’t have the struggle with my forum bookmarks and likes - and I would be way more flexible because of categories for these.

But this is my opinion and a very very good reason for such a Macro.

Greetings from Germany :de:

Tobias

1 Like

I wrote a long series about sqlite3 and Keyboard Maestro a while back:

The beauty of using sqlite3 is that it's included in macOS, and Apple uses it extensively in their apps, so it's unlikely to vanish any time soon.

Yes, you can set their storage location (depending on how the macro is written; check out my Quick Web Search macro as an example of one that lets you specify a storage location.

But no, not on dropbox. sqlite3 is not written with multiple simultaneous access from networked machines in mind. It uses a file locking mechanism that doesn't work across a network, amongst other issues.

I don't think I want to try to address synching, given the complexities involved. But it wouldn't be hard to write a macro that made sure the database was quit on other Macs, then copied the current file from one Mac to others. But syncing changes and keeping them current both directions? Not something I'm even going to attempt :).

-rob.

2 Likes

Thanks for the good food for thought, Kevin!

The problem is the stiction involved in getting the data to the storage locations, and finding one that works the way my brain works. Plain text doesn't do it for me, because I like categorization. Eagle Filer is the best I've come across so far, for the way I work, but still, I have to launch a separate app.

And it's tough to beat hitting a single key combo, waiting a few seconds, and having everything I need safely stored away in a database.

No, you also need the name of the macro, which you won't get from either the page title or the URL.

Data retrieval is much simpler from a database than from a pure text file: "I want the names and titles of the macros I saved in 2023." Figuring out how to do that for text is a bit of a search and regex nightmare, at least for me. With a database, it's one bit of SQL.

Apple relies heavily on sqlite3 for its own apps, and I've never had a single issue with the databases I use in my macros. Yes, it's much less portable than pure text, but honestly, portability isn't high on my scale of importance: I only use Macs, and if I ever stopped, the data in this particular instance would be worthless anyway.

And if I did want the data, exporting from mysqlite3 as pure text is, again, just one bit of SQL.

sqlite3 is built into every Mac, and given Apple's reliance on it, will be for some years to come.

Given that the data being collected is only of use to Keyboard Maestro, I don't see it as being weird to use Keyboard Maestro to collect information about Keyboard Maestro macros.

I completely agree, which is why I was curious as to how many might find this particular system useful to them—I know it will be useful to me, which is why I'm continuing to develop it. But if it wasn't of interest to others, I'd be much less thorough with how I approach certain aspects of it (the interface, ability to export data, move the stored file, etc.).

Thank you for the thoughtful feedback!

-rob.

2 Likes

Thank you @griffman and @Nr.5-need_input. It seems that you are both avid collectors of macros! I can understand how you would want a dedicated solution.

1 Like

Yep, very important thing to watch for, and that's one of those bits I wouldn't pay any attention to if I were writing this only for my own use. There are two ways to solve it in a macro I share with others:

The right way is to get a collection of language examples for "Copy Link" in a number of languages, then let the user specify their preferred language, and press the key that way.

However, you can't press a key with a variable, so that would require AppleScript. And even worse is what's shown in your screenshot: In German, "L" wouldn't be enough, because there are four other items that start with "L", so the first one would be picked; the macro would have to type out "Link k" in order to find it uniquely. And that's six calls to the AppleScript. Ugh.

So I'll have to consider other solutions for the language issue, and I have some ideas on that.

That's definitely already on my list; I just didn't bother with the proof of concept.

Thanks for the feedback!

-rob.

1 Like

...and

You could always split the difference. Record everything in a CSV plain text file, but search that CSV via SQL. We've had a couple of examples on the Forum of using mysql on the fly with a CSV input -- from @drdrang, IIRC, and maybe others.

The portability of plain text, (most of) the functionality of SQL, without -- unless you've really gone wild with bookmarking! -- too much extra overhead.

Updating seems like it'd be a major PitA, though.

-rob.

Not too bad, tbh -- read in the CSV, update via mysql, export the (now updated) CSV.

My personal preference would be to use a database -- but I can see where @kevinb's coming from, especially when it comes to sharing between machines.

Hmmm... A web-hosted DB, API to easily submit new bookmarks, usual form-based front end for searching and updating (starts doodling on a piece of paper)...

Yea, I thought about that. Then I remembered why I got tired of running macosxhints :).

-rob.

2 Likes