How to sort a list by first non-modifier character?

August · June 6, 2024, 1:06am

I am trying to sort a list of triggers, all are hotkeys, all or nearly all have some combination of modifiers along with the key name, except the F-keys may or may not have modifiers.

I see exactly the sort that I want in KBM when I go to All Macros and sort by the second column. Then all the various modifier+A hotkeys are sorted together, followed by the modifier+B hokeys, and so on.

If I have a list of those as text, how can I sort by the first non-modifier key in the list item?

(Seeing this list is making it obvious to me that I have a bunch of obsolete macros to clean up, but that's not what I'm asking for help with — unless you have magic tricks.)

Airy · June 6, 2024, 1:23am

How do you get that list as a text variable? I find that statement confusing. If you have that list stored as a text variable, then we can use the shell sort command to solve the problem.

Also, don't you mean "how can I sort by the first modifier key", since they are already sorted by the first non-modifier key? E.g., the first non-modifier keys in those shortcuts are A, A, A, A, A, B, B, C, C, C, C, D, E, and that's exactly how they are already sorted. But the first modifier keys are Shift, Option, Control, Control, Option, Control, etc.

August · June 6, 2024, 3:11am

No, the image is an example of my desired sort. The image is how KBM can sort macros by their first trigger. I am trying to sort a different list, e.g. a list of file names that all have modifier characters in the names, and get it to use the same kind of sorting as this example, sorting first by the first character after the modifier characters.

griffman · June 6, 2024, 4:33am

The unix sort command can sort on key fields if you can separate them with delimiters. So one admittedly hacky solution would involve building your list like this:

Item 99##A
Item 3##B
Alpha Item 1#⇧#A
Words here#⌘⌃#B
More Words#⌃⇧#A
Foobar Bazbin#⌃#B

Put that in a variable in the shell, $myvar or whatever, then you can sort it like this....

EDIT: My bad, can't use echo with a multi-line variable (though it works in this case). Should be like this...

printf "%s\n" "$myvar" | sort -t'#' -k3,3

$ printf "%s\n" "$myvar" | sort -t'#' -k3,3
Alpha Item 1#⇧#A
Item 99##A
More Words#⌃⇧#A
Foobar Bazbin#⌃#B
Item 3##B
Words here#⌘⌃#B

The -t parameter sets the delimiting character, and -k is the key field to sort on (3,3 just means sort only on field 3. If you want them by letter then alpha by title within the letter, you'd add a second sort:

$ printf "%s\n" "$myvar" | sort -t'#' -k3,3 -k1,1
Alpha Item 1#⇧#A
Item 99##A
More Words#⌃⇧#A
Foobar Bazbin#⌃#B
Item 3##B
Words here#⌘⌃#B

Then you could just search and replace the # to spaces or whatever to get your final formatting. Definitely hacky, and I'm sure there's a much better way (or 100 better ways), but it was kind of fun to come up with :).

-rob.

ComplexPoint · June 6, 2024, 5:08am

Leaving aside the keys for secondary and tertiary sort order, you can, of course, write sortBy and sortOn functions in JavaScript, passing to them a function which picks out the feature to sort on.

Very roughly, for example:

Expand disclosure triangle to view JS source

(() => {
    "use strict";

    // main :: IO ()
    const main = () => {
        const km = Application("Keyboard Maestro");

        return sortOn(
            x => x[1]
        )(
            hotkeysAndMacroNames(km)
        )
        .map(x => x.join("\t"))
        .join("\n");
    };


    // hotkeysAndMacroNames :: [(Modifier, Key, Name)]
    const hotkeysAndMacroNames = app => {
        const
            macros = app.macros,
            names = macros.name();

        return zipWith(
            name => modKey => 0 < modKey.length
                ? [modKey.concat(name).flat()]
                : []
        )(names)(
            macros.triggers
            .description().map(
                ts => 0 < ts.length
                    ? [keyParts(ts)]
                    : []
            )
        )
        .flat()
        .filter(x => 3 === x.length);
    };


    const keyParts = descriptions => {
        const
            iKey = descriptions.findIndex(
                s => s.startsWith("The Hot Key")
            );

        return -1 === iKey
            ? []
            : modsAndLetter(
                descriptions[iKey].slice(12, -10)
            );
    };


    const modsAndLetter = keyString =>
        [...both(
            x => x.join("")
        )(
            span(c => "⇧⌃⌥⌘".includes(c))([
                ...keyString
            ])
        )];


    // --------------------- GENERIC ---------------------

    // Tuple (,) :: a -> b -> (a, b)
    const Tuple = a =>
    // A pair of values, possibly of
    // different types.
        b => ({
            type: "Tuple",
            "0": a,
            "1": b,
            length: 2,
            *[Symbol.iterator]() {
                for (const k in this) {
                    if (!isNaN(k)) {
                        yield this[k];
                    }
                }
            }
        });


    // both :: (a -> b) -> (a, a) -> (b, b)
    const both = f =>
        // A tuple obtained by separately
        // applying f to each of the two
        // values in the given tuple.
        ([a, b]) => Tuple(
            f(a)
        )(
            f(b)
        );


    // comparing :: Ord a => (b -> a) -> b -> b -> Ordering
    const comparing = f =>
    // The ordering of f(x) and f(y) as a value
    // drawn from {-1, 0, 1}, representing {LT, EQ, GT}.
        x => y => {
            const
                a = f(x),
                b = f(y);

            return a < b
                ? -1
                : a > b
                    ? 1
                    : 0;
        };


    // sortBy :: (a -> a -> Ordering) -> [a] -> [a]
    const sortBy = f =>
        // A copy of xs sorted by the comparator function f.
        xs => xs.slice()
        .sort((a, b) => f(a)(b));


    // sortOn :: Ord b => (a -> b) -> [a] -> [a]
    const sortOn = f =>
        // Equivalent to sortBy(comparing(f)), but with f(x)
        // evaluated only once for each x in xs.
        // ('Schwartzian' decorate-sort-undecorate).
        xs => sortBy(
            comparing(x => x[0])
        )(
            xs.map(x => [f(x), x])
        )
        .map(x => x[1]);


    // span :: (a -> Bool) -> [a] -> ([a], [a])
    const span = p =>
    // Longest prefix of xs consisting of elements which
    // all satisfy p, tupled with the remainder of xs.
        xs => {
            const i = xs.findIndex(x => !p(x));

            return -1 !== i
                ? Tuple(
                    xs.slice(0, i)
                )(
                    xs.slice(i)
                )
                : Tuple(xs)([]);
        };


    // zipWith :: (a -> a -> b) -> [a] -> [b]
    const zipWith = f =>
    // A list with the length of the shorter of
    // xs and ys, defined by zipping with a
    // custom function, rather than with the
    // default tuple constructor.
        xs => ys => xs.slice(
            0, Math.min(xs.length, ys.length)
        )
        .map((x, i) => f(x)(ys[i]));


    // main :: IO ()
    return main();
})();

August · June 6, 2024, 10:12am

Thanks @ComplexPoint, that looks interesting and possible, but I think I'll go with Rob's (@griffman) suggestion. From here it seems like what it would take for me to wrap my head around that much JavaScript would be a bit daunting, and I've used JavaScript before (but 20 years ago).

Thanks Rob @griffman, I had been thinking along similar lines myself. I think it's a straightforward way to do it that is based on the essential patterns of the problem. It's a pretty simple regex substitution to surround any sequence of modifier symbols with some other Unicode symbol to use as a field delimiter. I think I'll use "÷", the division sign, because visually it's distinct and there's zero chance of it being in the input strings.

The tricky entries are those that have no modifiers. One possibility is that the unmodified key name is F1, F2, ... etc. That's a separate regex search and replace, just like the first. But if that field is empty, how do I find it? I guess I'll have to have something specific from the other content to look for, which means this can't be an all-purpose subroutine if it handles those cases. In my first versions, that is probably OK, I can modularize the components rather than the whole thing, so if I reuse this in a different context, I only have one section to change.

To see what I want to use this for, take a look at Feature Requests: Comments on Triggers, Rearranging Triggers .

Nige_S · June 6, 2024, 10:24am

OK, here's my stab at it. The theory is that we inject null characters around the modifiers, do a shell sort using null as the delimiter, then strip the nulls from the result (just in case!).

Sort by Char then Mods.kmmacros (4.4 KB)

Image

There are issues -- it "breaks" with unmodified hot keys, sorting those to the top, and will put modified Fn keys between E and G, but you could probably fix those by munging the list before sorting. Whether that's worth doing will depend on your hot key choices!

August · June 6, 2024, 10:38am

Wow, thanks!

It's already past 3:30am here in California, so testing this will have to wait until tomorrow night, at the soonest.

griffman · June 6, 2024, 12:56pm

Building on @Nige_S' solution (brilliant choice to use the null char as the delimiter!), here's my take, which handles (I think) all the possible cases by simply processing each line one by one. So single characters, function keys, and normal hot keys should all sort properly.

I also wondered if you really need to retain the "The Hot Key..." and "is pressed" bits, as if you're building a custom string, all you might really want is the sorted list of triggers. So there's a yes/no variable at the top that you can switch to get either output :).

And yea, looping through things is archaic and slow, but the list isn't that long, and the KM engine is quite fast. I pasted in 50 lines as a test, and it took 0.4 seconds.

EDIT: Replaced macro due to my early-morning inability to count.

Download Macro(s): hot key sorting.kmmacros (25 KB)

Macro screenshot

Macro notes

Macros are always disabled when imported into the Keyboard Maestro Editor.
- The user must ensure the macro is enabled.
- The user must also ensure the macro's parent macro-group is enabled.

System information

macOS 14.5
Keyboard Maestro v11.0.3

This is definitely not the most efficient way to do this, but it seems functional. I don't know if there are trigger key variations that I've missed, though.

-rob.

Nige_S · June 6, 2024, 2:04pm

I think you've left a 1 in your F key test -- perhaps replace with ^The Hot Key F\d+?

griffman · June 6, 2024, 2:09pm

I did it that way on purpose, because every function key shortcut will start with F1. It might just be F1, or anything up to F19, but I'm not aware of any keyboards with F20+? But yes, your method would be a better match condition, and much more future proof.

The actual regex in that section should be fine, as it just groups F with whatever characters follow it until the space.

-rob.

Nige_S · June 6, 2024, 2:17pm

Even F2?

griffman · June 6, 2024, 2:19pm

Sigh. What I get for coding too early! Ya, you're right I'm wrong :). Macro replaced above, so future visitors will get the right one, and wonder what this was all about.

-rob.

Nige_S · June 6, 2024, 4:35pm

OK -- here's a complete version, based on what I understand @August's needs to be (from here).

It grabs the list of triggers from a macro selected in the KM Editor and sorts them into the following order:

Hot key Fn triggers sorted by number (so F9 is before F10) and sub-sorted by modifiers
All other hot key triggers, ASCII-sorted by key, sub-sorted by modifiers
All other triggers, in ASCII sort order of the triggers' descriptions -- this is not the same as you see in the GUI! For example, the This application: Finder: Activates is described as "Application 'Finder' activates", so sorts before "At system sleep".

It then rearranges (actually, adds copies of the triggers in order, then deletes the originals from the "front" of the trigger list) the triggers of the selected macro, leaving a (hopefully!) tidier list...

Please try this on copies of macros first -- for your own peace of mind, if nothing else. It's been extensively tested, but there's no way I could try it on every possible trigger combo...

Replace Triggers with Sorted Ones.kmmacros (12.3 KB)

Image

I know there are improvements to be made -- for example, I've left in the repeat loop to delete "old" triggers one-by-one so you can see what's happening, but it would be better to

delete triggers 1 thru (count of triggerList) of the selectedMacro

...and nuke them in one operation. I look forward to seeing what other changes you all make!

August · June 7, 2024, 5:55am

Wow! @Nige_S, Rob @griffman, you guys are awesome. What an amazing collaboration to witness. I'm glad that I was able to inspire you (although that was not my intent).

From everything that I can tell by reading your notes and skimming through the script, it does exactly what I was hoping to create, and much more — I was going to limit myself to single click hotkeys with modifiers or F-keys and that's it. Do what I need for now but make it expandable if ever it's necessary.

Would you elaborate on what is so brilliant, other than the fact that it can be counted on to never appear in a trigger string?

Absolutely correct, the text description padding is superfluous, I left that in just to see what the script-in-progress was doing. You are correct, all that matters is the sorting of the list and keeping the original index with each entry as they are reordered. So at this point I'm not sure what the optional format does, or is it gone now?

Archaic? You would rather have it recursively defined?

griffman · June 7, 2024, 6:00am

I kept trying to find an ASCII character that was guaranteed to not be in your textual descriptions, but it seemed tough. So then I tried emoji and Unicode, but those have to be coded differently in the shell command, and I didn't know the format (and didn't want to futz around finding it out). The null is brilliant because it won't ever be in a name, and it doesn't even show up visually.

The final macro is all @Nige_S, and I assume he didn't include an option. I just left it in my demo in case you needed those words for some reason.

No, I just meant that proceeding row by row isn't the "perfect" solution when there are ways to manage the entire text block—but those ways are beyond my skill level, so it's not necessarily archaic to me :).

-rob.

August · June 8, 2024, 12:45am

Hmm. I ge the following error in the KBM Engine Log. Because I'm not yet fully understanding the script, I'm not yet making sense of where it comes from.

2024-06-07 17:36:09 Execute an AppleScript failed with script error: text-script:599:605: execution error: Can’t make text item 2 of item 1 of {"", "⌥⌘. is pressed==mySep==22", "⌥⌘1 is pressed==mySep==31", "⌥⌘2 is pressed==mySep==26", "⌥⌘3 is pressed==mySep==27", "⌥⇧⌘3 is pressed==mySep==28", "⌥⇧⌘4 is pressed==mySep==25", "⌥⇧⌘5 is pressed==mySep==24", "⌥⇧⌘7 is pressed==mySep==23", "⌥⌘A is pressed==mySep==1", "⌥⌘B is pressed==mySep==2", "⌥⇧⌘B is pressed==mySep==32", "⌥⌘C is pressed==mySep==29", "⌥⇧⌘C is pressed==mySep==30", "⌥⌘D is pressed==mySep==3", "⌥⇧⌘D is pressed==mySep==4", "⌥⌘E is pressed==mySep==5", "⌥⌘F is pressed==mySep==6", "⌥⌘G is pressed==mySep==7", "⌥⌘H is pressed==mySep==33", "⌥⌘K is pressed==mySep==8", "⌥⇧⌘K is pressed==mySep==9", "⌥⌘L is pressed==mySep==10", "⌥⌘M is pressed==mySep==11", "⌥⌘O is pressed==mySep==12", "⌥⌘P is pressed==mySep==13", "⌥⌘Q is pressed==mySep==14", "⌥⌘R is pressed==mySep==15", "⌥⌘S is pressed==mySep==16", "⌥⌘T is pressed==mySep==17", "⌥⌘U is pressed==mySep==18", "⌥⌘V is pressed==mySep==19", "⌥⌘X is pressed==mySep==20", "⌥⇧⌘X is pressed==mySep==21"} into type number. (-1700). Macro “Replace Triggers with Sorted Ones” cancelled (while executing Execute AppleScript).

Or with linebreaks:

2024-06-07 17:36:09 Execute an AppleScript failed with script error: text-script:599:605: execution error:
 Can’t make text item 2 of item 1 of {"", "⌥⌘. is pressed==mySep==22",
 "⌥⌘1 is pressed==mySep==31", "⌥⌘2 is pressed==mySep==26",
 "⌥⌘3 is pressed==mySep==27", "⌥⇧⌘3 is pressed==mySep==28",
 "⌥⇧⌘4 is pressed==mySep==25", "⌥⇧⌘5 is pressed==mySep==24",
 "⌥⇧⌘7 is pressed==mySep==23", "⌥⌘A is pressed==mySep==1",
 "⌥⌘B is pressed==mySep==2", "⌥⇧⌘B is pressed==mySep==32",
 "⌥⌘C is pressed==mySep==29", "⌥⇧⌘C is pressed==mySep==30",
 "⌥⌘D is pressed==mySep==3", "⌥⇧⌘D is pressed==mySep==4",
 "⌥⌘E is pressed==mySep==5", "⌥⌘F is pressed==mySep==6",
 "⌥⌘G is pressed==mySep==7", "⌥⌘H is pressed==mySep==33",
 "⌥⌘K is pressed==mySep==8", "⌥⇧⌘K is pressed==mySep==9",
 "⌥⌘L is pressed==mySep==10", "⌥⌘M is pressed==mySep==11",
 "⌥⌘O is pressed==mySep==12", "⌥⌘P is pressed==mySep==13",
 "⌥⌘Q is pressed==mySep==14", "⌥⌘R is pressed==mySep==15",
 "⌥⌘S is pressed==mySep==16", "⌥⌘T is pressed==mySep==17",
 "⌥⌘U is pressed==mySep==18", "⌥⌘V is pressed==mySep==19",
 "⌥⌘X is pressed==mySep==20", "⌥⇧⌘X is pressed==mySep==21"} into type number.
 (-1700). Macro “Replace Triggers with Sorted Ones” cancelled
 (while executing Execute AppleScript).

I think it's reporting the whole contents of the list and saying that the script wants item 1 of that list, which is "" and the script is then trying to get text item 2 of *that" empty string, and then attempting to turn that "text item 2 of item 1" into a number, but it can't, because there is no item 2 of item 1 because item 1 is empty. (I probably could have said that more concisely if I actually knew what it was that I was trying to describe.)

I haven't yet figured out where in the script the blank entry is coming from.

Nige_S · June 8, 2024, 8:28am

You're getting an extra, blank, line from somewhere in your hot keys list. KM then sorts that to the top of the output text, and when AS converts that to a list with every paragraph of... you get an empty string as the first item -- the "" after {.

Since AS can't get text item 2 of an empty string it throws an error.

Quickest fix is probably to drop in a "Filter: Trim Whitespace" action between the last two actions of the macro:

August · June 8, 2024, 8:48am

Yes, that would probably do it, for this case, but I really would like to understand where that blank entry is coming from when it is not happening for you. The Triggers in my target KBM macro seem normal, at least to me. An error like that makes this not publishable, by my standards, because there's no indication that the patch works in other cases when I have no idea what those cases might be. E.g., if I knew that the blank line was an artifact of how the list was originally generated, and I didn't want to muck around with generating the list in a different way, then I'd be confident that the patch was reasonable. But it didn't happen in your testing, so it's not that simple.

I think my next step is to print out the the contents of the list at various stages, using Display Text in WIndow actions, and see if I can find how early in the process it shows up.

Thanks

Nige_S · June 8, 2024, 10:06am

The reason is the usual one -- because I'm stupid!

It's this action:

...which works fine when each variable has a value. But if one is empty then that token evaluates to an empty string and you get a blank line. For example, for a macro with one F1 and one When Finder activates trigger only you'd get

Hot Key F1...

When Application Finder...

...and the AS will barf on that middle line after it's converted to an empty string list item.

The best fix is to not worry about the empty strings when building the text, but to deal with it within the final AS action. Rather than rebuild the list without the blanks we can test eachTrigger before adding and keep a count of items skipped so our later delete has the right number of repeats (which also means we can remove the "Filter: Trim" action).

Full final AS would be:

set inst to system attribute "KMINSTANCE"
tell application "Keyboard Maestro Engine"
	set triggerList to getvariable "Local_theText" instance inst
end tell

tell application "Keyboard Maestro"
	set selectedMacro to macro id (item 1 of (get selectedMacros))
	set triggerList to every paragraph of triggerList
	set AppleScript's text item delimiters to "==mySep=="
	set emptyCount to 0
	repeat with eachTrigger in triggerList
		if contents of eachTrigger is not "" then
			set triggerIndex to text item 2 of eachTrigger as number
			copy (get trigger triggerIndex of selectedMacro) to end of selectedMacro's triggers
		else
			set emptyCount to emptyCount + 1
		end if
	end repeat
	repeat ((count of triggerList) - emptyCount) times
		delete first trigger of selectedMacro
	end repeat
end tell

Complete adjusted macro:

Replace Triggers with Sorted Ones v2.kmmacros (12.3 KB)

Image

How to sort a list by first non-modifier character?

Options