How to sort a list by first non-modifier character?

griffman · June 6, 2024, 12:56pm

Building on @Nige_S' solution (brilliant choice to use the null char as the delimiter!), here's my take, which handles (I think) all the possible cases by simply processing each line one by one. So single characters, function keys, and normal hot keys should all sort properly.

I also wondered if you really need to retain the "The Hot Key..." and "is pressed" bits, as if you're building a custom string, all you might really want is the sorted list of triggers. So there's a yes/no variable at the top that you can switch to get either output :).

And yea, looping through things is archaic and slow, but the list isn't that long, and the KM engine is quite fast. I pasted in 50 lines as a test, and it took 0.4 seconds.

EDIT: Replaced macro due to my early-morning inability to count.

Download Macro(s): hot key sorting.kmmacros (25 KB)

Macro screenshot

Macro notes

Macros are always disabled when imported into the Keyboard Maestro Editor.
- The user must ensure the macro is enabled.
- The user must also ensure the macro's parent macro-group is enabled.

System information

macOS 14.5
Keyboard Maestro v11.0.3

This is definitely not the most efficient way to do this, but it seems functional. I don't know if there are trigger key variations that I've missed, though.

-rob.

Nige_S · June 6, 2024, 2:04pm

I think you've left a 1 in your F key test -- perhaps replace with ^The Hot Key F\d+?

griffman · June 6, 2024, 2:09pm

I did it that way on purpose, because every function key shortcut will start with F1. It might just be F1, or anything up to F19, but I'm not aware of any keyboards with F20+? But yes, your method would be a better match condition, and much more future proof.

The actual regex in that section should be fine, as it just groups F with whatever characters follow it until the space.

-rob.

Nige_S · June 6, 2024, 2:17pm

Even F2?

griffman · June 6, 2024, 2:19pm

Sigh. What I get for coding too early! Ya, you're right I'm wrong :). Macro replaced above, so future visitors will get the right one, and wonder what this was all about.

-rob.

Nige_S · June 6, 2024, 4:35pm

OK -- here's a complete version, based on what I understand @August's needs to be (from here).

It grabs the list of triggers from a macro selected in the KM Editor and sorts them into the following order:

Hot key Fn triggers sorted by number (so F9 is before F10) and sub-sorted by modifiers
All other hot key triggers, ASCII-sorted by key, sub-sorted by modifiers
All other triggers, in ASCII sort order of the triggers' descriptions -- this is not the same as you see in the GUI! For example, the This application: Finder: Activates is described as "Application 'Finder' activates", so sorts before "At system sleep".

It then rearranges (actually, adds copies of the triggers in order, then deletes the originals from the "front" of the trigger list) the triggers of the selected macro, leaving a (hopefully!) tidier list...

Please try this on copies of macros first -- for your own peace of mind, if nothing else. It's been extensively tested, but there's no way I could try it on every possible trigger combo...

Replace Triggers with Sorted Ones.kmmacros (12.3 KB)

Image

I know there are improvements to be made -- for example, I've left in the repeat loop to delete "old" triggers one-by-one so you can see what's happening, but it would be better to

delete triggers 1 thru (count of triggerList) of the selectedMacro

...and nuke them in one operation. I look forward to seeing what other changes you all make!

August · June 7, 2024, 5:55am

Wow! @Nige_S, Rob @griffman, you guys are awesome. What an amazing collaboration to witness. I'm glad that I was able to inspire you (although that was not my intent).

From everything that I can tell by reading your notes and skimming through the script, it does exactly what I was hoping to create, and much more — I was going to limit myself to single click hotkeys with modifiers or F-keys and that's it. Do what I need for now but make it expandable if ever it's necessary.

Would you elaborate on what is so brilliant, other than the fact that it can be counted on to never appear in a trigger string?

Absolutely correct, the text description padding is superfluous, I left that in just to see what the script-in-progress was doing. You are correct, all that matters is the sorting of the list and keeping the original index with each entry as they are reordered. So at this point I'm not sure what the optional format does, or is it gone now?

Archaic? You would rather have it recursively defined?

griffman · June 7, 2024, 6:00am

I kept trying to find an ASCII character that was guaranteed to not be in your textual descriptions, but it seemed tough. So then I tried emoji and Unicode, but those have to be coded differently in the shell command, and I didn't know the format (and didn't want to futz around finding it out). The null is brilliant because it won't ever be in a name, and it doesn't even show up visually.

The final macro is all @Nige_S, and I assume he didn't include an option. I just left it in my demo in case you needed those words for some reason.

No, I just meant that proceeding row by row isn't the "perfect" solution when there are ways to manage the entire text block—but those ways are beyond my skill level, so it's not necessarily archaic to me :).

-rob.

August · June 8, 2024, 12:45am

Hmm. I ge the following error in the KBM Engine Log. Because I'm not yet fully understanding the script, I'm not yet making sense of where it comes from.

2024-06-07 17:36:09 Execute an AppleScript failed with script error: text-script:599:605: execution error: Can’t make text item 2 of item 1 of {"", "⌥⌘. is pressed==mySep==22", "⌥⌘1 is pressed==mySep==31", "⌥⌘2 is pressed==mySep==26", "⌥⌘3 is pressed==mySep==27", "⌥⇧⌘3 is pressed==mySep==28", "⌥⇧⌘4 is pressed==mySep==25", "⌥⇧⌘5 is pressed==mySep==24", "⌥⇧⌘7 is pressed==mySep==23", "⌥⌘A is pressed==mySep==1", "⌥⌘B is pressed==mySep==2", "⌥⇧⌘B is pressed==mySep==32", "⌥⌘C is pressed==mySep==29", "⌥⇧⌘C is pressed==mySep==30", "⌥⌘D is pressed==mySep==3", "⌥⇧⌘D is pressed==mySep==4", "⌥⌘E is pressed==mySep==5", "⌥⌘F is pressed==mySep==6", "⌥⌘G is pressed==mySep==7", "⌥⌘H is pressed==mySep==33", "⌥⌘K is pressed==mySep==8", "⌥⇧⌘K is pressed==mySep==9", "⌥⌘L is pressed==mySep==10", "⌥⌘M is pressed==mySep==11", "⌥⌘O is pressed==mySep==12", "⌥⌘P is pressed==mySep==13", "⌥⌘Q is pressed==mySep==14", "⌥⌘R is pressed==mySep==15", "⌥⌘S is pressed==mySep==16", "⌥⌘T is pressed==mySep==17", "⌥⌘U is pressed==mySep==18", "⌥⌘V is pressed==mySep==19", "⌥⌘X is pressed==mySep==20", "⌥⇧⌘X is pressed==mySep==21"} into type number. (-1700). Macro “Replace Triggers with Sorted Ones” cancelled (while executing Execute AppleScript).

Or with linebreaks:

2024-06-07 17:36:09 Execute an AppleScript failed with script error: text-script:599:605: execution error:
 Can’t make text item 2 of item 1 of {"", "⌥⌘. is pressed==mySep==22",
 "⌥⌘1 is pressed==mySep==31", "⌥⌘2 is pressed==mySep==26",
 "⌥⌘3 is pressed==mySep==27", "⌥⇧⌘3 is pressed==mySep==28",
 "⌥⇧⌘4 is pressed==mySep==25", "⌥⇧⌘5 is pressed==mySep==24",
 "⌥⇧⌘7 is pressed==mySep==23", "⌥⌘A is pressed==mySep==1",
 "⌥⌘B is pressed==mySep==2", "⌥⇧⌘B is pressed==mySep==32",
 "⌥⌘C is pressed==mySep==29", "⌥⇧⌘C is pressed==mySep==30",
 "⌥⌘D is pressed==mySep==3", "⌥⇧⌘D is pressed==mySep==4",
 "⌥⌘E is pressed==mySep==5", "⌥⌘F is pressed==mySep==6",
 "⌥⌘G is pressed==mySep==7", "⌥⌘H is pressed==mySep==33",
 "⌥⌘K is pressed==mySep==8", "⌥⇧⌘K is pressed==mySep==9",
 "⌥⌘L is pressed==mySep==10", "⌥⌘M is pressed==mySep==11",
 "⌥⌘O is pressed==mySep==12", "⌥⌘P is pressed==mySep==13",
 "⌥⌘Q is pressed==mySep==14", "⌥⌘R is pressed==mySep==15",
 "⌥⌘S is pressed==mySep==16", "⌥⌘T is pressed==mySep==17",
 "⌥⌘U is pressed==mySep==18", "⌥⌘V is pressed==mySep==19",
 "⌥⌘X is pressed==mySep==20", "⌥⇧⌘X is pressed==mySep==21"} into type number.
 (-1700). Macro “Replace Triggers with Sorted Ones” cancelled
 (while executing Execute AppleScript).

I think it's reporting the whole contents of the list and saying that the script wants item 1 of that list, which is "" and the script is then trying to get text item 2 of *that" empty string, and then attempting to turn that "text item 2 of item 1" into a number, but it can't, because there is no item 2 of item 1 because item 1 is empty. (I probably could have said that more concisely if I actually knew what it was that I was trying to describe.)

I haven't yet figured out where in the script the blank entry is coming from.

Nige_S · June 8, 2024, 8:28am

You're getting an extra, blank, line from somewhere in your hot keys list. KM then sorts that to the top of the output text, and when AS converts that to a list with every paragraph of... you get an empty string as the first item -- the "" after {.

Since AS can't get text item 2 of an empty string it throws an error.

Quickest fix is probably to drop in a "Filter: Trim Whitespace" action between the last two actions of the macro:

August · June 8, 2024, 8:48am

Yes, that would probably do it, for this case, but I really would like to understand where that blank entry is coming from when it is not happening for you. The Triggers in my target KBM macro seem normal, at least to me. An error like that makes this not publishable, by my standards, because there's no indication that the patch works in other cases when I have no idea what those cases might be. E.g., if I knew that the blank line was an artifact of how the list was originally generated, and I didn't want to muck around with generating the list in a different way, then I'd be confident that the patch was reasonable. But it didn't happen in your testing, so it's not that simple.

I think my next step is to print out the the contents of the list at various stages, using Display Text in WIndow actions, and see if I can find how early in the process it shows up.

Thanks

Nige_S · June 8, 2024, 10:06am

The reason is the usual one -- because I'm stupid!

It's this action:

...which works fine when each variable has a value. But if one is empty then that token evaluates to an empty string and you get a blank line. For example, for a macro with one F1 and one When Finder activates trigger only you'd get

Hot Key F1...

When Application Finder...

...and the AS will barf on that middle line after it's converted to an empty string list item.

The best fix is to not worry about the empty strings when building the text, but to deal with it within the final AS action. Rather than rebuild the list without the blanks we can test eachTrigger before adding and keep a count of items skipped so our later delete has the right number of repeats (which also means we can remove the "Filter: Trim" action).

Full final AS would be:

set inst to system attribute "KMINSTANCE"
tell application "Keyboard Maestro Engine"
	set triggerList to getvariable "Local_theText" instance inst
end tell

tell application "Keyboard Maestro"
	set selectedMacro to macro id (item 1 of (get selectedMacros))
	set triggerList to every paragraph of triggerList
	set AppleScript's text item delimiters to "==mySep=="
	set emptyCount to 0
	repeat with eachTrigger in triggerList
		if contents of eachTrigger is not "" then
			set triggerIndex to text item 2 of eachTrigger as number
			copy (get trigger triggerIndex of selectedMacro) to end of selectedMacro's triggers
		else
			set emptyCount to emptyCount + 1
		end if
	end repeat
	repeat ((count of triggerList) - emptyCount) times
		delete first trigger of selectedMacro
	end repeat
end tell

Complete adjusted macro:

Replace Triggers with Sorted Ones v2.kmmacros (12.3 KB)

Image

August · June 8, 2024, 4:32pm

Thanks for identifying where the problem comes from and it makes sense. You were testing all three types of trigger and my first use of the macro only had one type, resulting in the blank line.

But I take issue with your description of the source of the problem. As you have no doubt noticed before, and it is certainly true in this case, developing a macro of this complexity is an iterative process, you go around and around and around. That is why collaboration so often helps, as you and Rob demonstrated yesterday.

You don’t do me any service, or your self, Rob, or anyone who encounters this thread in the future, by calling yourself “stupid” for not having thought of every use case in a single iteration. Maybe “cocky“ for calling it “complete”, but certainly not stupid, as demonstrated by the solutions to many, many problems in the development of this macro.

I have lots of sympathy, empathy, and compassion for the tendency to preemptively self criticize. I have a bad case of it myself.

Airy · June 8, 2024, 4:42pm

I'm 10x more stupid than Nigel. And while self criticism isn't always the best thing, if it's facetious, it shouldn't be criticized.

August · June 8, 2024, 4:52pm

I disagree. I am a firm believer that when code reflects the actual structure of the problem, it is cleaner, more robust, and easier to read and maintain. There is nothing in the nature of the problem which inherently requires counting the number of items that have been skipped.

I think the issue is that this action, which you identified above, tries to do too much at once.

The problem is the extra line feeds within this action that are still included in the resulting text even if one or more of these variables is empty.

Based on my aesthetic of following the structure of the problem, I would replace that action with a series of three tests which would each only add the associated variable if it was not empty. Nothing extra added, nothing to count or remove later.

I can probably do that tonight, using your earlier version.

Nige_S · June 8, 2024, 5:35pm

Since my exact thought, when I realised what the problem was, was "Oh Nige, you stupid tw*t...", I thought it only fair to include such (suitably sanitised for the Forum). I should, perhaps, have included a smiley since I'm not exactly beating myself up about it

Yes, I can see that that is a good idea. I did consider:

Your approach. Very valid, but a big change to the macro and also patterns I assume you're familiar with
Keep as is then "Search and Replace" the blank lines out just before passing to the AS. Sort of "this line intentionally blank" when a trigger class is empty, which may help with debugging. A smaller change and, again, patterns I assume you're familiar with
The way I did it. A quick and easy change that a) keeps the problem in, useful if you want to step through with the debugger to see what's happening, and b) solves it with an AS pattern you may not have seen before

So I went with option 3 -- not the best solution, but a fun fix for the Forum.

TBH, the whole AppleScript should be redone to include error checking, a roll-back if the trigger replacement fails part way, etc. But I'm bad at that kind of "looking to the future" error checking, never knowing what to include, and just wait til things blow up and then add something to cope if the same happens again. A perk of only writing these things for myself -- I should probably smarten up for Forum submissions...

August · June 9, 2024, 9:03am

So you did it the “tricky” way because he wanted to show off a technique that might be useful elsewhere. OK, I can understand that. I will dig deeper to make sure that I follow your trick.

Since all the input comes from KBM, I think we can presume that it’s been well checked by Peter. Trusting the input, we only need to make sure that the macro is invoked in the right circumstances, i.e., with a macro selected that has at least one trigger. I could be wrong.

Of course the repeated admonition to make sure you only run this macro on a copy of your working macro should help this somewhat. I imagine if you wanted to set up a failsafe you could check if the name of the selected macro ends in “copy“ and if not if there exists a macro with an identical name which ends in “copy”, only then run the macro. A safety check like this might be very useful as a subroutine in many circumstances to only allow the running macro to modify another macro if a copy of the target macro already exists.

Anyway, I'll dig into your "Option 3b" trick and come back with any questions.

Thanks

August · June 11, 2024, 5:24am

Hi @Nige_S,

I have to agree that your way is actually simpler; it certainly takes up less screen real estate.. To not add extra linefeeds requires a lot of IF-tests, as shown below.

The AppleScript to handle the extra lines is pretty straighforward. From your macro and you exposition above:

Nige_S:

. . .
		if contents of eachTrigger is not "" then
			set triggerIndex to text item 2 of eachTrigger as number
			copy (get trigger triggerIndex of selectedMacro) to end of selectedMacro's triggers
		else
			set emptyCount to emptyCount + 1
		end if
	end repeat
	repeat ((count of triggerList) - emptyCount) times
. . .

Nige_S · June 14, 2024, 11:44am

I'm not so sure. We can assume that built-in actions handle errors "gracefully", but it's our responsibility to make sure we're passing in "sane" data. And when it comes to script actions -- and also to KM sub-routines we write -- we should really be adding error-handling as well, treating them like black-box functions.

I've already demonstrated how an incorrect assumption about input data can lead to problems!

I think this can be done a lot simpler -- you only need a couple of "If" actions, then use "Filter: Trim Whitespace" to handle any leading/trailing linefeeds:

Even more simply, we could build the text as we did originally and S'n'R for blank lines:

(For only 3 variables you can use "Search for string: \n\n; Replace with: \n" -- the regex makes it usable for 4 or more variables where multiple consecutive lines might be blank.)

And the best way is probably to do both the sanitising before passing the data to the AS and the error-checking within the script itself -- trust, but verify!

August · June 14, 2024, 9:28pm

Thanks. I agree. I like your simplification of the IF-Then structures too. Does the Filter, Trim Whitespace, remove blank lines in the middle of the list? The only definition I've found is

Trim Whitespace. (Remove leading and trailing whitespace from a string.)

and that suggests not.

And I appreciate your notes about using more than three sorting groups. I had already been thinking about that, wanting to put the groups in the order in which I think about them, so that means alphabetic first, then numbers, then punctuation, then Fn keys.

Sorting number separately from punctuation requires separating shifted numbers from unshifted numbers, so that's four groups. In either method that requires another regex search and another sort. In my structure, that requires yet another IF-Then layer in assembling the sorted lists but yours does not.

How to sort a list by first non-modifier character?

Options