How to sort a list by first non-modifier character?

[August]

@Nige_S

It's this action:

Thanks for identifying where the problem comes from and it makes sense. You were testing all three types of trigger and my first use of the macro only had one type, resulting in the blank line.

But I take issue with your description of the source of the problem. As you have no doubt noticed before, and it is certainly true in this case, developing a macro of this complexity is an iterative process, you go around and around and around. That is why collaboration so often helps, as you and Rob demonstrated yesterday.

You don’t do me any service, or your self, Rob, or anyone who encounters this thread in the future, by calling yourself “stupid” for not having thought of every use case in a single iteration. Maybe “cocky“ for calling it “complete”, but certainly not stupid, as demonstrated by the solutions to many, many problems in the development of this macro.

I have lots of sympathy, empathy, and compassion for the tendency to preemptively self criticize. I have a bad case of it myself.

[Airy]

@August

I have lots of sympathy, empathy, and compassion for the tendency to preemptively self criticize.

I'm 10x more stupid than Nigel. And while self criticism isn't always the best thing, if it's facetious, it shouldn't be criticized.

[August]

@Nige_S

The best fix is to not worry about the empty strings when building the text

I disagree. I am a firm believer that when code reflects the actual structure of the problem, it is cleaner, more robust, and easier to read and maintain. There is nothing in the nature of the problem which inherently requires counting the number of items that have been skipped.

I think the issue is that this action, which you identified above, tries to do too much at once.

The problem is the extra line feeds within this action that are still included in the resulting text even if one or more of these variables is empty.

Based on my aesthetic of following the structure of the problem, I would replace that action with a series of three tests which would each only add the associated variable if it was not empty. Nothing extra added, nothing to count or remove later.

I can probably do that tonight, using your earlier version.

[Nige_S]

@August

But I take issue with your description of the source of the problem.

Since my exact thought, when I realised what the problem was, was "Oh Nige, you stupid tw*t...", I thought it only fair to include such (suitably sanitised for the Forum). I should, perhaps, have included a smiley since I'm not exactly beating myself up about it

@August

I disagree. I am a firm believer that when code reflects the actual structure of the problem, it is cleaner, more robust, and easier to read and maintain.

Yes, I can see that that is a good idea. I did consider:

1. Your approach. Very valid, but a big change to the macro and also patterns I assume you're familiar with
2. Keep as is then "Search and Replace" the blank lines out just before passing to the AS. Sort of "this line intentionally blank" when a trigger class is empty, which may help with debugging. A smaller change and, again, patterns I assume you're familiar with
3. The way I did it. A quick and easy change that a) keeps the problem in, useful if you want to step through with the debugger to see what's happening, and b) solves it with an AS pattern you may not have seen before

So I went with option 3 -- not the best solution, but a fun fix for the Forum.

TBH, the whole AppleScript should be redone to include error checking, a roll-back if the trigger replacement fails part way, etc. But I'm bad at that kind of "looking to the future" error checking, never knowing what to include, and just wait til things blow up and then add something to cope if the same happens again. A perk of only writing these things for myself -- I should probably smarten up for Forum submissions...

[August]

@Nige_S

      and b) solves it with an AS pattern you may not have seen before

So I went with option 3 -- not the best solution, but a fun fix for the Forum.

So you did it the “tricky” way because he wanted to show off a technique that might be useful elsewhere. OK, I can understand that. I will dig deeper to make sure that I follow your trick.

@Nige_S

include error checking

Since all the input comes from KBM, I think we can presume that it’s been well checked by Peter. Trusting the input, we only need to make sure that the macro is invoked in the right circumstances, i.e., with a macro selected that has at least one trigger. I could be wrong.

@Nige_S

a roll-back if the trigger replacement fails part way

Of course the repeated admonition to make sure you only run this macro on a copy of your working macro should help this somewhat. I imagine if you wanted to set up a failsafe you could check if the name of the selected macro ends in “copy“ and if not if there exists a macro with an identical name which ends in “copy”, only then run the macro. A safety check like this might be very useful as a subroutine in many circumstances to only allow the running macro to modify another macro if a copy of the target macro already exists.

Anyway, I'll dig into your "Option 3b" trick and come back with any questions.

Thanks

[August]

Hi @Nige_S,

@August

I'll dig into your "Option 3b" trick and come back with any questions.

I have to agree that your way is actually simpler; it certainly takes up less screen real estate.. To not add extra linefeeds requires a lot of IF-tests, as shown below.

The AppleScript to handle the extra lines is pretty straighforward. From your macro and you exposition above:

@Nige_S

. . .
		if contents of eachTrigger is not "" then
			set triggerIndex to text item 2 of eachTrigger as number
			copy (get trigger triggerIndex of selectedMacro) to end of selectedMacro's triggers
		else
			set emptyCount to emptyCount + 1
		end if
	end repeat
	repeat ((count of triggerList) - emptyCount) times
. . .

[Nige_S]

@August

Since all the input comes from KBM, I think we can presume that it’s been well checked by Peter.

I'm not so sure. We can assume that built-in actions handle errors "gracefully", but it's our responsibility to make sure we're passing in "sane" data. And when it comes to script actions -- and also to KM sub-routines we write -- we should really be adding error-handling as well, treating them like black-box functions.

I've already demonstrated how an incorrect assumption about input data can lead to problems!

@August

To not add extra linefeeds requires a lot of IF-tests, as shown below.

I think this can be done a lot simpler -- you only need a couple of "If" actions, then use "Filter: Trim Whitespace" to handle any leading/trailing linefeeds:

Even more simply, we could build the text as we did originally and S'n'R for blank lines:

(For only 3 variables you can use "Search for string: \n\n; Replace with: \n" -- the regex makes it usable for 4 or more variables where multiple consecutive lines might be blank.)

And the best way is probably to do both the sanitising before passing the data to the AS and the error-checking within the script itself -- trust, but verify!

[August]

@Nige_S

And the best way is probably to do both the sanitising before passing the data to the AS and the error-checking within the script itself -- trust, but verify!

Thanks. I agree. I like your simplification of the IF-Then structures too. Does the Filter, Trim Whitespace, remove blank lines in the middle of the list? The only definition I've found is

• Trim Whitespace. (Remove leading and trailing whitespace from a string.)

and that suggests not.

And I appreciate your notes about using more than three sorting groups. I had already been thinking about that, wanting to put the groups in the order in which I think about them, so that means alphabetic first, then numbers, then punctuation, then Fn keys.

Sorting number separately from punctuation requires separating shifted numbers from unshifted numbers, so that's four groups. In either method that requires another regex search and another sort. In my structure, that requires yet another IF-Then layer in assembling the sorted lists but yours does not.

[Airy]

@August

and that suggests not.

You make a good point there, but I just tested it, and it does remove blank lines.

[tiffle]

Really? - check this out:

@peternlewis

OK, so you are wanting the blank lines in the middle removed, which is not what the Trim Whitespace does (it removes white space from either end of the string.

Use a Search and Replace action, with regex \R+ and replace \n and that will replace any sequence of one or more line endings with a single linefeed.

[August]

@Airy

it does remove blank lines.

Very curious...

@peternlewis

OK, so you are wanting the blank lines in the middle removed, which is not what the Trim Whitespace does (it removes white space from either end of the string.

Since @peternlewis has documented the ends-only behavior and also provided (at the linked comment) a S&R Regex example for the middles, I wouldn't count on the undocumened behavior persisting. Peter may even treat that as a bug.

[August]

@Nige_S

use "Search for string: \n\n; Replace with: \n"

Nige, the comment from @peternlewis on another thread, linked above, suggests searching for "\R+" and replacing it wih "\n". The definition of "\R" is:

Match a new line character, or the sequence CR LF. The new line characters are \u000a, \u000b, \u000c, \u000d, \u0085, \u2028, \u2029.

While not necessary, because you are controlling the %LINEFEED% characters that are added, I think the \R would be more robust through latter edits and mods of the macro that a less experienced user might make.

[Nige_S]

@August

Nige, the comment from @peternlewis on another thread, linked above, suggests searching for "\R+" and replacing it wih "\n"...

... While not necessary, because you are controlling the %LINEFEED% characters that are added...

That's why I went with \n rather than \R. We're putting in the extra characters, we know the extra characters we need to remove. Our "future editor" will have to actively break things by replacing the keyboard strokes in that action with %Return% tokens, replacing the entire field with text from an old-style EOL Mac text editor or even from (shudder) Windows, or similar.

I'd rather have it break in such circumstances rather than cater for all eventualities!

Better to handle such things with error-checking in the AS, IMO. Conversion to list using every paragraph is line-ending agnostic (which is why it is used so often) so will this problem would be caught by our "skip an empty list item" routine. (There's even an argument, continuing the "only remove what we know we added" theme, of changing that to "split to list using linefeed".)

Which leaves "future editors" that change our macro so that the variables are concatenated with \x00, an emoji, nowYouAreFubared, or something equally weird -- you can only go so far in saving people from themselves!

[peternlewis]

@Airy

You make a good point there, but I just tested it, and it does remove blank lines.

The Filter action with Trim Whitespace trims white space from the start of the string, and white space from the end of the string.

It does not remove blank lines.

[ComplexPoint]

i.e.

\n\nalpha beta gamma\n\n

→ alpha beta gamma

but

alpha\n\nbeta\n\ngamma

remains unchanged.

---

In my experience this is a very general use of the term trim, across a variety of different languages and libraries.

It always refers to the start (and/or) end of strings,
never to anything in the middle of them.

[Airy]

@peternlewis

The Filter action with Trim Whitespace trims white space from the start of the string, and white space from the end of the string.

It does not remove blank lines.

What I meant when I spoke was "it does remove blank lines from the beginning and end of the string." I guess I could have been clearer. I just triple checked, and it does remove those blank lines, but not blank lines in the middle.