Damn! Was my cunning ploy that transparent?
2 Likes
Making progress…
Capture now works in all those browsers. And wow, was that an interesting exercise into "how differently can we all do the exact same thing?"
I should have an updated testable version later today.
Part of my testing was trying to find a way around this problem. I thought JavaScript would be the solution at first, but I couldn't get one script to work in any more than a couple of the browsers, then I'd have to tweak it, and maybe it'd work in a couple more. Some I never got anything.
In the end, the contextual menu is the most consistent way to get the download link across all the above browsers.
So now there's a "user must translate" section of variables where you'll enter the "Copy Link" words in your native language as shown in each browser's contextual menu. This works because you can send typed text to the contextual menu, so it's possible to "type" the entire string to select the proper menu item.
-rob.
2 Likes
Here's the second alpha release. For now, the working name is KMF Macro Collector, but I hate that, so it will change. Do not think this is in any way ready for production—the database may change, the variables may change, and there's a whole lot more to do and fix.
__ KMF Macro Collector Macros.kmmacros (151 KB)
But if you'd like to test, I'd love some feedback. What's new in this version:
• User-settable locations for the database and (optional) local macro library.
• A new download option to download a copy of a macro when you save it.
• Support for Arc, Brave, Firefox(1), Google Chrome, Microsoft Edge(1), Orion, Safari, and Vivaldi.
• A sanity check to warn you if you're adding a non macro. Please let me know if you find a posted macro that fails this check—that is, you're told it's not a macro but it is.
• The macro has been split into many smaller macros to make it much easier for me to update and work on.
• Non-English lanuages may be supported—I definitely need testing help here!
(1) In both Firefox and Microsoft Edge, I could not find any way to select the text of the link. In every other browser, it's automatically selected when you right-click. But not in those two. If you use Firefox or Edge, you'll have to first manually select just the text of the link, then put your mouse somewhere over the selected text, then invoke the macro. I searched quite a bit, but didn't find any easy, simple, reliable way to select the text of the link.
How to use
Open the 10 - Set key variables macro. You have to set the first three values for the database location (full path), database name (no extension!), and storage location for downloaded macros (full path).
The orange box below is for non-English systems. In the box for each browser, enter the words used for the "Copy Link" text in the contextual menu. While I could have combined many of these (as they're the same), I wasn't sure if all devs would translate them the same, so I left them separate.
With the local language version of Copy Link inserted in each box, the macro should work as expected (fingers crossed).
Once you've set the locations for the files, open your browser and find a macro download link in the forums. Hover over it and (unless using Firefox or Edge) press Control-C. From there, follow the prompts :). Edge and Firefox users need to first select the text of the link, then invoke the macro after putting the mouse somewhere over the selected text.
The following items are saved for each macro:
Date added, macro name, macro link, forum thread link, author name (optional), category, description, local path (for downloaded macros), and a UUID
Not all of this stuff is used yet, but it will be :).
All the work today went into the "add" side of the macro; the "browse" side is basically unchanged from yesterday.
-rob.
1 Like
Hi @griffman, thanks for the info on sqlite3. I have made a modified version of your Macro to store the data in a Keyboard Maestro Dictionary which I write and read to Dropbox as I want to be able to access from more than one Mac. That's a solution that has worked well for me with a number of Macros without problems over the years.
I like the concept of your Macro - thanks again for sharing!
1 Like
off-topic but lanuages does sound exotic.
1 Like
Hello Rob (@griffman)
Many thanks for posting this - damn your update was quick …
I’ve flagged it to my automatic downloads system and it’s maybe going to be installed later this day.
But I am very busy right now so the first testing I could accomplish are maybe not before Thursday or Friday this week. But I’ll give my very best to reply as quick as possible, this time - I Promise (I’m sorry about forgetting giving you more details on your Web Search Macro.)
Greetings from Germany 
Tobias
If you have any interest in sharing, I'd love to take a look. I honestly don't do a lot with dictionaries, because (1) I hate how "out of sight" they are (they should be browsable, just like variables are browsable), and (2) I find it immensely difficult to build data structures within them that make sense to my mind and that I can figure out how to query. For example, finding things that meet multiple different criteria, etc.
-rob.
Good Stuff.
I set the 3 'green key variables'.
I added 3 different macros and links - each in it's own category.
When I trigger "00 - Browse macro collection" I see the 3 individual links I added.
Should I not be able to 'browse' the categories at that point?
As noted above, I've done absolutely nothing with the browser side of things. What's there is there :).
-rob.
1 Like
Yes, I agree with all of that, the dictionaries are quite opaque. But I use @DanThomas's Dictionary Editor to look inside them and remove entries etc.
And I have been using the Dictionaries when I need to store multiple bits of information with a key to call them up.
Yes, the KM Dictionaries need a "key" to call the info rather than being able to search by multiple criteria.
So, my approach to the present situation is to grab all the bits of info using some of the methods in your macro (name of macro, web address to download, link to page the macro is on, my own description etc). Then choose which of those bits of information should be the key (I chose the "name of the macro + its author" in a prompt field that I can edit). And save that info as a multi-line dictionary entry.
So, a KM Dictionary entry might look like (each item on new line):
Post Title as text
URL link to the post
The title I have saved this dictionary entry as (and used for Dictionary key) as text
URL link to download the Macro directly
My own description of the Macro as text
Date Added (today's date) as text
Author of the Macro as text
Then, in the Browser Macro I use a Prompt with List just as you have done.
The chosen item in the prompt (the macro name and "key") is used to open that multiline Dictionary entry and save it to a Local Variable, in my case called Local_Data
And now the individual lines of the Variable can easily be accessed using Keyboard Maestro custom delimiter of \n (as each entry is on a different line).
So I get something line this:
Post Title: %Variable%Local_Data[1]\n%
Post Link: %Variable%Local_Data[2]\n%
Save As: %Variable%Local_Data[3]\n%
Macro Link: %Variable%Local_Data[4]\n%
Macro Description: %Variable%Local_Data[5]\n%
Date Added: %Variable%Local_Data[6]\n%
Author: %Variable%Local_Data[7]\n%
I can view all the items and access whichever one I want by its line number. So, if I want the URL to download the Macro it would be %Variable%Local_Data[4]\n%
In order to save the Dictionary to Dropbox I use a write to text Action to write the Dictionary to a text file on Dropbox. This same file gets read at the start of the Macro.
To do that I use @DanThomas's method to read and write Dictionaries to text files
I can put the Macros here (or send to you by a private message). I didn't want to just load it up here as it is as it might muddy-up this thread. And the method of using Dictionaries that are read and written to files is something I have used and am comfortable with, but could be confusing.
Thanks for that ... and related to this new macro, your post inspired me to...
...write a database lock controller to enable the database file to live on iCloud, etc. I had a realization that I could make it quite simple, as the macro only needs access to the actual database file briefly while it's running. So I've created my own locking mechanism that prevents simultaneous use of the macro on multiple Macs.
That should take care of any corruption issues, but I'm going to play around with it extensively before I upload a build with that feature enabled.
-rob.
1 Like
That sounds great. And could be useful for so many things beyond this particular Macro too.
1 Like
Here's build 0.5.7…
__ KMF Macro Collector - v0.5.7 Macros.kmmacros (277 KB)
…with a bunch of new stuff:
- Reworked section on adding categories and authors.
- Brand-new HTML table view of collection.
- Live filtering, sorting in table browsing view.
- All macro metadata input is now done in one dialog.
- Experimental network support added: The database will lock if in use. I hope.
- Browsing table is functional, still not done.
- New "onboarding" to handle initial setup. Please test by deleting the globals and your database before running.
- Categories and descriptions can be edited.
- The "Browse" macros can be run from anywhere.
The big news is the new HTML table view:
It's nowhere near done and a bit clunky now, but it does work. Very basic help: The two iconic columns—one either opens the folder with the downloaded macro or downloads the macro; the other opens the forum thread with the macro in it. Click an entry to edit its description or category. That part is very clunky.
Note I changed the activation keys to control-option-E, as you can now run the table from anywhere.
Feedback, please, if you try it—I don't expect everything to work, but it's working better now than I thought it would be at this point.
-rob.
1 Like
It's looking good!
Setup is obviously a bit fiddly at the moment but I got there.
- Main comment is that the path to the two folders (database and downloaded macros) will not work if the username is different on two Macs which makes syncing with Dropbox difficult. i.e. I would need a tilde path like this to work:
~/My Dropbox/Macro Collector Macros
-
If the collector macro is run before the mouse is hovering over the macro link to download, strange things happen. I wonder if rather the automating the copying of the links by replicating the right-click menu it might be safer to ask the user to copy the download link to the clipboard manually by right-clicking and copying the link?
-
It is good to have the Author as one of the bits of data stored. I have been trying to think if there is any way to grab this data automatically from the thread.
-
Funnily enough (and this just relates to my tests with a KM Dictionary) you calling your latest Macro __ KMF Macro Collector - v0.5.7 Macros.kmmacros messed up the prompt with list as KM treats double underscores at the start of entries in a special way... when I tried to add this Macro to my own KM Dictionary the "key" was saved as the macro name __ KMF Macro Collector - v0.5.7 Macros.kmmacros but the prompt with list dropped the __ at the start which meant I couldn't access my entry with the prompt from list... I had to add in a Search and Replace to change "__" at the start of any Macro grabbed to "-"
In case it is of interest here is how my own version of your idea is progressing. I have one hotkey to call the Macro and then choose whether to Collect or Browse. In the video I first collect a Macro and then run again to Browse.
1 Like
It's in progress :). I'd never tried having a macro open the settings macro for editing before. And while it works, if this is popular enough, a standard Choose File dialog would be preferable. But at the moment, not sure there's enough interest to merit the time; we'll see.
Are you talking about if you have the macro synced automatically across Macs? The problem is going to be that not everyone's shared folders are in their home path, which means it can't just make that assumption. However, I could change the macro to look for a tilde-path and then get its expanded version and use that as the variable's value. (I shy away from tilde-paths because there seem to be places they fail miserably; full paths are foolproof, unless you have different user names on different Macs with the same shared folder 
).
What should happen is that it would simply stop; that's not working for you? Regardless, the next version has a separate solution to that problem.
I wanted something as easy for me to use as possible, removing the overhead of saving stuff. That's why I wrote it to do the selection and copying, so I can just hit a hot key and have stuff happen. So I'm unlikely to change this, though I suppose if other feedback is strong enough, it could be an on/off option in setup.
I spent quite a bit of time on it, and the only solution is to use JavaScript, at a level I cannot write. I asked a couple LLMs, and they came up with viable models, but ... this then really limits the browser support, because they have to work with Keyboard Maestro's JavaScript in Front Browser. And the user has to have enabled JS support for external apps in each browser, etc.
In the end, because the macro builds a pop-up menu of authors over time, I didn't feel it was worth the effort.
That finally inspired me to do a mass renaming, which I've been meaning to do for a long time, because of the double-underscore issue. So the next version won't have that problem :).
Thanks for the feedback!
-rob.
1 Like
Count me as a user but not a developer, as this is above my Keyboard Maestro pay grade. I will contribute a perspective.
My Keyboard Maestro Finder Folder has just over 13K items in it. A good bit of this is category or subject folders where I attempt to sort out the mountain of macros and Keyboard Maestro's workings to both make it easier to find a macro or idea AND as part of my learning path, which for me at the initial stage is a matter of being able to create categories and subcategories to anchor items within.
From the perspective of not having a lot of distinctions around coding, I have to translate many things into my own language. From this, I find how information is displayed to play a crucial part.
So my interest in this project is from both the convenience of a highly tailored capture tool AND as a new display and retrieval option for the information that might in itself forward my learning.
1 Like
That would be very helpful to me.
Looks great by the way.
Cheers
1 Like
Yes, that's all I would need. The path to my Dropbox folder is the same from User onwards. It's just that when I have set up my Macs I sometimes have my name in a slightly different form - sometimes just first name, sometimes first name and surname etc (I tried in the past to be consistent but after discovering ~/ works for username I have got lazy...).
1 Like
No new test, today, because things have gone backwards pretty far…but in the end, it will be for the better.
-rob.
3 Likes
Well, that took a bit longer than expected :). But here's version 0.7, which is a complete rewrite of the "gather" side of the macro. (I have not spent any time on the browsing side since the last update.)
— KMFᴀᴄᴛs - v0.7 Macros.kmmacros (394 KB)
Yea, new name, too—KMFacts. There's a corny name behind the "acts" part, but it's irrelevant. Other than the name, here's what's changed in other areas.
The big news is that the author information is now automatically collected when you run the macro…and you can save it incredibly quickly, if you want to skip categorizing and commenting on entries:
An animated GIF hides within
This is all courtesy of...
- A super-fast super-awesome JavaScript that collects all macros and authors in one shot, and returns it in SQL-structured text, ready to add to the database. Sweet! No more weird click-select stuff at all! I wrote none of this, but spent probably three or four hours working with Claude to get it just right.
Here are the other changes...
- Downside of the above is that Arc and Firefox are no longer supported, as you can't run JS in them from Keyboard Maestro. That leaves six browsers where it seems to work very well.
- You can now save topics (discussions) as well as macros, or both.
- A Turbo-add button adds topic and all macros with a few keystrokes (activate macro, 'x', Enter) in about a second.
- That new interface for selecting macros and threads to save uses snippets, so requires Keyboard Maestro v11. Sorry, I'm not going back—I love how fast and keyboard-centric the UI is with Snippets. (I could do custom HTML prompts for all that, but that's a ton of work for very little reward.)
- Oops, I lied: I did change one thing on the browse site: New buttons let you see Topics, Macros, or Both.
- In theory, tilde-paths should work now, so as long as your shared folder's local homes are in your user's folder, you should be able to use network locations for your data.
- Added a second table to the database to greatly ease new record additions.
That's a lot of stuff, and it all works really nicely. Launch the macro on any KM Forums page, and you'll see this box appear nearly instantly, as seen in the GIF:
By the time the box appears, all that data is already stored in a (temporary) sqlite3 table. If you just want to collect data in a hurry, without doing any categorization, that first box is for you: Put an 'x' (or really, any character other than Space) in that box, and the macro will add everything it can to the database when you click OK, as seen in the GIF I posted above.
Oh yea, and because it adds the data to the database right away, by the time that dialog appears, the system knows if you've already added any of the items in that topic to your collection:
So you won't ever add a duplicate, because you can't.
Alternatively, put an 'x' (or whatever) in the other boxes, and the OK button will then walk through each item, letting you assign a category and add descriptions as you go.
How to test
You'll have to remove your old database, as the new collection method required a new table. Sorry, but hopefully you weren't using this for real stuff yet! (It's possible this will happen again with future updates, until it's released. Don't use this for mission critical stuff yet!).
Please also delete any old versions of the macro before installing this one.
Enter your paths in the settings file—you can use the "Run me first" macro if you want to, or just enter them directly. Tilde paths should be supported.
My network record locking seems to be working well—I've had no corruption despite trying my best on two Macs at once. The writes happen so quickly that the database is hardly ever in a dangerous state. What I'm less sure about is the effect of time delays on network sync—if you make a slew of changes on machine 1, then quickly move to machine 2 and make another slew of changes before Dropbox can catch up ... what happens?
I honestly don't know the answer to that question, but for that potential reason, network use will probably remain experimental and best considered a sync method versus a simulatenous-use thing, even with my basic locker in place.
Next steps
The output side needs serious attention. It needs an advanced search feature. And here's the biggie: I'm going to remove the Library feature. Trying to keep everything in sync will be a nightmare, and as I've been using the tool, I've realized it's not how I'd use it in practice. There will be a download button, but downloaded items will just go to your Downloads folder, and the database won't try to stay in sync with what you have and have not downloaded. (I'm not even going to record that a download happened; there's no real upside to that.) I need to add my macro update checker, as this macro will probably get fairly regular updates.
I expect I've got at least a couple more weeks of work before it's ready, but it's getting there!
Feedback
Please try to break this—find weird forum threads, try things I didn't expect. I can only do so much on my own, and would welcome some feedback on things that are borked!
-rob.
5 Likes