The %Return% token

[peternlewis]

@JMichaelTX

\R will work for KM 8+ and macOS 10.10+

\R will work in macOS 10.11+, in (I believe) any version of Keyboard Maestro.

@JMichaelTX

KM \n will match only LF in a Search and Search/Replace.

Correct.

@JMichaelTX

BBEdit 12+ will match either CR or LF with a \n

BBEdit uses \n LF line endings internally, despite whatever the line endings of the text file are. It converts on load/save to the line endings of the file on disk. You can see this by changing the line ending popup, saving to disk, and using both File ➤ Hex Dump Front Window and File ➤ Hex Dump File. The former always shows \n (0x0A), while the latter shows whatever is configure. And it seems that its regex has somehow been configured such that \r and \n both match the internal \n characters. \r will match line endings, and \n will match line endings, and \r\n will match only if there is two line endings in a row (ie, a blank line). Someone from Bare Bones would have to comment to know more for sure.

And no, I wont be messing around with the regex in Keyboard Maestro, \r is CR, \n is LF, \R is the best thing to use in Keyboard Maestro as long as you'll never see OS X 10.10 (which kind of makes me wish I had made Keyboard Maestro 8 require 10.11+), otherwise (?:\r\n|\r|\n) or the equivalent is necessary.

[JMichaelTX]

@Tom

So, I think it would be a good idea to place a big fat warning on the KM Wiki article:

“Do not use the %Return% token for anything, except you really need it for some obscure reason or legacy texts. Do never use it when feeding the variable to the shell! Use \n or %LineFeed% instead.”

I agree Tom.

@peternlewis, any objections or comments?

If not, I will make the changes, and the ones from my corrected summary, to:
token:Return [Keyboard Maestro Wiki]

[JMichaelTX]

@peternlewis

\R will work in macOS 10.11+, in (I believe) any version of Keyboard Maestro.

Thanks. I have corrected my OP.

[peternlewis]

No there should not be a “big fat warning” not use %Return% or \r.

There can be a note to ensure you know which line endings you want to use, and that generally scripts and files will need %LineFeed% or \n, while clipboards probably want %Return% or \r, and anything you write should generally try to handle either.

With minor amendments, the info in your Summary previously is appropriate for the token pages.

[JMichaelTX]

@peternlewis, would you agree that it is better to use a LF unless you have good reason to do otherwise, and always use LF in Shell Scripts?

[peternlewis]

Nope, I use %Return% frequently (I use it in 37 different macros). I don’t use %LineFeed% anywhere.

%Return% is generally fine in any macro except when dealing with variables that will be used in shell scripts.

For Insert Text by Typing or Pasting, or for setting the clipboard, %Return% is fine, and probably more correct than %LineFeed%.

[Tom]

My recommendation with the “big fat warning” was based on the following assumption and scenario:
‌

• There are quite some users who don’t know that there different types of line breaks.
• It is not uncommon to use Shell Script actions, even for new users.

Now…

1. If such a user comes across the Wiki article he will most certainly opt for the %Return% token, since it sounds more familiar than “LineFeed” (also the Return key is named “Return” and not “Line Feed”)
2. When the variable value with the %Return% token gets passed to the Shell Script action the action most likely will fail.
3. The user has no clue why it is failing.
   ‌

The other way round:

When a user is using the %LineFeed% token (or \n) in cases where — according to your explanations — the %Return% token would be “more correct”, chances that something will fail are rather low; at least lower.

Indeed, since I’m using KM I literally never used the %Return% token or \r and, off the cuff, I can’t remember a single problem because of that.

---

That being said, I appreciate your above explanations where and why a CR would be more correct. Didn’t know that.

[ccstone]

Hey Folks,

Personally I advocate using linefeeds everywhere, since that’s the UNIX way of doing things.

I only use alternative end-of-line characters when genuinely needed.

Points-to-note:

Mixed end-of-line characters (CR, LF, CRLF) are where macOS and Apps tend to hiccough most often.

BBEdit

Until version 12 came out just recently BBEdit used carriage returns internally — even when it was set to use other end-of-line characters — and it would convert them appropriately as necessary.

There were a number of instances (particular when scripting) where you might unexpectedly get carriage returns, and this as @Tom has noted can cause shell-scripts a ton of grief.

\n and \r are synonymous in BBEdit’s search and replace engine. In BBEdit 12 they stand solely for linefeed.

If you want to find a carriage return in BBEdit you must use the Hex or Unicode representation:

\x0D
\x{000D}

It’s sad that after 16+ years of OSX we’re still having this discussion…

(In other words — the issue hasn’t been completely sorted out by Apple.)

-Chris

[Tom]

@ccstone

If you want to find a carriage return in BBEdit you must use the Hex or Unicode representation:

\x0D
\x{000D}

Somehow this doesn’t work for me (BBEdit 12.0.2)

1. New document
2. Save it with CR linebreaks (“Legacy Mac”)
3. Paste a text with CRs into it
4. Save
5. Search for \x0D or \x{000D}

It only finds \x0A (LF), except the last CR which is preserved. File hex dump (as shown here) shows CRs.

[JMichaelTX]

TLDR:  Skip to "Conclusions" at bottom of post

@peternlewis

Nope, I use %Return% frequently (I use it in 37 different macros). I don't use %LineFeed% anywhere.

That is very interesting. I don't use either %Return% or %LineFeed% in any of my production macros (have used both in testing), and I've never encountered an issue or negative feedback from other users.

So I really have not found any need to use %Return%. I have used the Type Keystorke with a RETURN key press often, and it always works. This is usually in the context of choosing a menu item, or creating a new line in an app. Perhaps KM is sending an ASCII 13 (0x0D) in this case to the app?

KM Editor Uses LF

It is also interesting to note that the KM Editor seems to use LF when a manual RETURN key is pressed in text areas of KM Actions.

For example, using this source string, which I typed into the Set Variable Action:

and put it on the Clipboard, then did a Search & Replace with:

The results were

SEARCH AND REPLACE of Clipboard For \n
(_LF_ will appear at EOL for all matches)

Test in Keyboard Maestro 8.0.3 (8.0.3) on macOS 10.11.6._LF_
_LF_
Line with MANUAL RETURN_key_LF_
Line with token for RETURN
_LF_
Line with token for LINEFEED_LF_
_LF_
END OF TEST

As you can see, a match for \n was found for every line, even the line where I had a %Return%, followed by a manual RETURN key press.

Copy From Web Page Uses LF

Another example of using LF is with this macro that I just created for a user:

I copied the source data from a KM forum page, and pasted into:

The \n was found and replaced with a comma ,

### Choosing App Menu Can Use Either CR or LF
Note: I did NOT use a manual RETURN at the end of these lines.
Using _either_ `\r` OR `\n` (alone) resulted in the desired/expected outcome.

<img src="/uploads/default/original/3X/d/e/de4b8221e019bd81c791f88c2efdd793e49d0d02.png" width="416" height="405">

Insert Text by Typing into App (Evernote)

RESULTS:

### Conclusions

1. KM uses LF in the Editor for RETURN key presses
2. Copy from Web page uses LF
3. KM Execute Shell Script _requires_ LF
3. BBEdit uses LF
4. RegEx101.com uses LF
1. After extensive testing, I still find little need for `%Return%`
2. When I do want to use it, I find it more convenient, less verbose, to use `\r`, OR, if it is a KM text area, just use manual RETURN key press.
3. **LF seems to work for me in 99% of my use cases.**

<img src="/uploads/default/original/2X/9/9fa034801953dfe7d1745afcb7087c7530a06e23.gif" width="70" height="17"  alt="updated" title="updated"> 2017-10-28 21:00 CT

**IMO, the best practice for most KM users** 

* When typing text in a KM Action and you need a new line, just press the RETURN key.
  * Don't use a token or symbol for CR or LF if don't really need to.
  * If they must enter a symbol (which should be rare) for new line, the use `\n` unless testing shows it does not work.  Then use `\r`.
* Use the `\R` in RegEx for matching new line characters (macOS 10.11+)
* Use `(?:\r?\n|\r)`  in RegEx for matching new line characters for macOS prior to 10.11

**Personally**, I will continue to use either manual RETURN key press OR `\R` everywhere in KM, unless that fails and I have a specific reason to use `\r`.
I have no use for either `%Return%` or `%LineFeed%`, as they both just require more typing, are verbose, and have no benefit over using `\r` or `\n`.

[peternlewis]

@JMichaelTX

I have used the Type Keystorke with a RETURN key press often, and it always works. This is usually in the context of choosing a menu item, or creating a new line in an app. Perhaps KM is sending an ASCII 13 (0x0D) in this case to the app?

You're talking Apples and Oranges here. The Return key simulates a keystroke, which is a key code. The resulting behaviour is dependent on the receiving application, it might OK a button, insert a CR or a LF or a CRLF, move to a different field, whatever.

@JMichaelTX

After extensive testing, I still find little need for %Return%

Some applications I have encountered have failed with LF line endings on the clipboard.

@JMichaelTX

When I do want to use it, I find it more convenient, less verbose, to use \r, OR, if it is a KM text area, just use manual RETURN key press.

Yes, but you know what a \r is.

The reason I typically use a %Return% is because it is much clearer to a much wider number of people, particularly where the data is a single line with a trailing line ending. Because you have to look very closely to see the difference between these two actions, but the third is obvious.

I tried every application I have open, and all worked with either %LineFeed% or %Return% on the clipboard.

@JMichaelTX

I have no use for either %Return% or %LineFeed%, as they both just require more typing, are verbose, and have no benefit over using \r or \n.

This is completely true for everyone who is fully familiar with \r and \n and other backslashed characters.

[ccstone]

@peternlewis

JMichaelTX:
I have no use for either %Return% or %LineFeed%, as they both just require more typing, are verbose, and have no benefit over using \r or \n.

This is completely true for everyone who is fully familiar with \r and \n and other backslashed characters.

Hey Guys,

I wouldn't quite go that far.

There are times and places where escaped single-letter-tokens are very hard to read and likely to be missed, so now and then the verbose form is preferable — even for those of us who've been using the short-form tokens for decades.

Peter — thanks for giving us choices.

-Chris

[JMichaelTX]

@peternlewis

You're talking Apples and Oranges here. The Return key simulates a keystroke, which is a key code. The resulting behaviour is dependent on the receiving application, it might OK a button, insert a CR or a LF or a CRLF, move to a different field, whatever.

Actually I'm not talking apples and oranges. I'm talking about what is easiest and most natural for most users. Using the Type Keystroke allows the user to do what is natural, without worrying about whether a CR or LF is generated. I submit that most users don't care whether a CR or LF is generated, they just want o get the job done.

The main point I have been trying to make, is that for most users, who want a new line in text, pressing the RETURN key is the most natural thing to do, and don't worry about what character code is generated.

If you want to activate a menu item which has been selected, then again, using a Type Keystroke with RETURN key is the most natural, easiest thing to do.

It is good to have the %Return% and %LineFeed% for those few occasions when the user may not want to show an actual new line in the UI, but to generate one in the text.

My other point is rarely does the user actually need to use the %Return% token, so let's not mislead the user into thinking they should do that by default.

[JMichaelTX]

@ccstone

If you want to find a carriage return in BBEdit you must use the Hex or Unicode representation:

\x0D
\x{000D}

@ccstone and @Tom:

Using \x0D worked for me, but it was a real challenge to get a CR into BBEdit.
Running BBEdit 12.0.1 (400010) on macOS 10.11.6.

I had to use this script:

set sourceStr to "LF" & linefeed & "CR" & return & linefeed
set the clipboard to sourceStr

Which produced this in BBEdit:

Then this Find worked:

[Tom]

@JMichaelTX

Using \x0D worked for me,

This works because your first line break is a LF.

Try these:

set sourceStr to "CR" & return & "CR" & return & "CR" & return & "CR" & return
set the clipboard to sourceStr

set sourceStr to "CR" & return & "LF" & linefeed & "CR" & return & "CR" & return
set the clipboard to sourceStr

set sourceStr to "CR" & return & "CR" & return & "CR" & return & "LF" & linefeed
set the clipboard to sourceStr

I only find \x0A (in the editor window).

[JMichaelTX]

@Tom

This works because your first line break is a LF.

Tom, I think the issue is that any line that ends with a CR is changed to a LF when it is pasted into BBEdit.

[Tom]

@JMichaelTX

I think the issue is that any line that ends with a CR is changed to a LF

I don’t think that is the issue. (That’s rather the expected behavior.) IMO the issue is that the CRs are not changed if the first line break is a LF. I tend to think that this is a bug (?)

[ccstone]

@Tom

This works because your first line break is a LF.

Try these:
...
I only find \x0A (in the editor window).

Hey Tom,

Hmm...

None of your scripts were able to place a carriage return into BBEdit (for me).

I just copied some text from Script Debugger (while making sure the first line was a linefeed) and pasted into BBEdit.

Then I deleted the linefeed, so I had only carriage returns.

BBEdit still found them with \x0D.

Feel free to send me a document that exhibits the problem off-list, and I'll be glad to test some more.

Oh...

You're not looking at the file with invisibles ON to determine if there actually ARE carriage returns are you.

So there ARE NO carriage returns to search for.

No wonder \x0D doesn't work for you, and you're only finding linefeeds with \x0A.

You're running into the issue that BBEdit 12 uses linefeeds INTERNALLY — just as it used to use carriage returns (I do believe).

-Chris

[Tom]

@ccstone

None of your scripts were able to place a carriage return into BBEdit (for me).

Yep, for me either. And that was the whole point of the scripts ;): to show that there is no way to get CRs into the BBEdit editor. With excactly one exception: the first line break is a line feed.

Those CRs are indeed found by \x0D. But those CRs shouldn’t be there, I think. Normally BBEdit doesn’t allow different types in one document, not in the editor, nor in the file on disk (check the Hex dump of your file).
In the editor it’s always LF; when it writes to file it writes either everything CR or everything LF, depending on the document setting. (Except with the above mentioned exception, the LF at the beginning. And that’s why I’m inclined to think this is a bug…)

And that’s also why I was surprised by your statement that you could find CRs with \x0D.
I thought you meant that with \x0D you could reveal the “real” CRs (as written to disk) behind their LF disguise in the editor — or something similar. (That’s why I said to Jim that the \x0D search worked only because he tricked BBEdit by sneaking in that LF at the beginning.)

But know I realize that with the \x0D search you referred precisely to those “bogus” CRs in the editor that BBEdit fails to LF-ize, not to the CRs in the file, or whatever I had thought.

Sorry for the misunderstanding.

[JMichaelTX]

@ccstone

If you want to find a carriage return in BBEdit you must use the Hex or Unicode representation:

Chris and @Tom:

Given the behavior of BBEdit, I don't think I'd use it to find/change CR in text, either from the Clipboard, or from a file.

It seems to be that the tools we can rely upon to do that are:

1. KM Search & Replace
2. JXA replace function
3. AppleScript using Satimage.osax change command.
4. Probably some unix tools you can use in a Shell Script, but I don't know them.

There are probably others, but those are the ones I'm familiar with.