I just spent quite some time debugging a macro for a forum member in a private thread, until I discovered by chance that he used the %Return% token to build a multi-line variable.
Of, course, when passing that variable for example to a sort Shell Script action it fails completely, since every Unix tool is expecting linefeeds, not carriage returns.
As far as I know, carriage returns were used only up to Mac OS 9.
So, I think it would be a good idea to place a big fat warning on the KM Wiki article:
“Do not use the %Return% token for anything, except you really need it for some obscure reason or legacy texts. Do never use it when feeding the variable to the shell! Use \n or %LineFeed% instead.”
BTW, in the same article %LineFeed% is misspelled as %Linefeed%.
Also, the KM Wiki shows the hex value for CR and LF:
The %LineFeed% token returns the linefeed (0x0A) character.
Does anyone find this useful?
It would seem to me that showing, and identifying, the ASCII decimal value and symbol would be more useful, given that most (I think) KM users are not C programmers, are maybe not programmers at all. AppleScript, JavaScript, and I think Perl all return the ASCII decimal value for these, using their native characterCode function.
So, I guess it's OK to show the hex value, but it should be last, and identified as such.
Maybe something like this:
The %LineFeed% token returns the linefeed character.
It is often referred to as LF, \n or just linefeed.
In a KM text block, you can use any of these:
a manual press of the RETURN key
\n
%LineFeed%
It is correct that the native Mac OS line endings were CR, and native Mac OS X unix susbsystems are LF line endings.
However, traditionally on the Mac, the clipboard uses Return (CR) characters even in OS X. That remained for many years after the transition to OS X (when native Classic Mac OS applications ran along side native Mac OS X applications). In fact there were many years when a LF on the clipboard was highly likely to cause problems.
At this point, when dealing with the clipboard, a CR is perfectly reasonably and likely more often handled correctly. Any modern Mac application should handle either CR or LF (or, frankly, CRLF), but that is not always the case.
When dealing with the shell or unix subsystems, LFs are definitely essential, so it is worth a mention on the Return and LineFeed token pages.
@JMichaelTX, your text looks fine, although what you get from a manual press of the Return key, I couldn't actually tell you - if it matters, you should ensure you get what you want. When sending text to the shell, Keyboard Maestro takes pains to translate the line endings appropriately regardless of the line endings in the text field.
So, if I understand correctly, you are saying for clipboard content that will be passed to Mac apps (not Unix programs) it is safer to use CRs than LFs?
Then we would have this situation:
%LineFeed%, \n :
…obligatory for variables that will be passed to the shell, or to any script. (Except AppleScript, which can handle both, I think.)
%Return%, \r :
…recommended for clipboards that will be passed to Mac apps.
That is fairly correct, although at the end of the day you’ll need to ensure you use the appropriate one in the appropriate place and there is no single correct answer.
As stated, yes, that would be my recommendation.
However there are almost certainly a few Mac applications that don’t work with CR line endings on the clipboard.
For shell scripts, LF will pretty much always be the correct. You can use the Keyboard Maestro Filter action to ensure the line endings are as desired.
Peter, I'm not sure what you mean by this. Are you saying that, somehow, the System Clipboard is changing LF characters into CR characters when they are copied to, or pasted from, the Clipboard?
That assertion is not supported by a number of tests that I have just done, using AppleScript, KM, BBEdit, Clipboard Viewer, and the RegEx101.com web site. (details later).
Actually, from what I am seeing, it looks like to me (subject to further testing and confirmation by others) that it is best to always use LF as the EOL (End of Line) character, rather than CR. I don't see any advantages of using CR (for Mac apps/data) over LF. Of course, if you are producing text targeted for Windows, then it is best to use both: CRLF.
Summary: Here is what I have found so far:
LF characters put on clipboard by AppleScript, and KM macro, remain as LF, as evidenced by reading the Clipboard by AppleScript, by pasting into BBEdit, and by viewing by the ClipBoard Viewer.app (not KM).
.
When I press the RETURN key, it generates a LF character in all of my apps (KM, AppleScript, BBEdit).
.
While the CR character is shown properly by the Clipboard Viewer.app, NONE of the CR characters are found by a \r search in BBEdit.
This could be due to BBEdit settings I have.
Further research is indicated.
.
Both LF and CR characters put on the Clipboard by AppleScript ARE found (matched) properly by a KM RegEx Search of Clipboard.
.
Major Issue: When text is pasted into RegEx101.com, ALL CRs are converted to LFs. I will report this to the developer.
.
Issue: When the KM token %Return% is at the end of a line with a manual RETURN key press, only ONE return is generated (should be two).
All of this needs to be confirmed by others.
Please post your results/comments.
### My Tests
### 1. All KM Using Both Tokens
<img src="/uploads/default/original/2X/6/68be0ad78c70c68a14863440c223a5cbf92b6147.png" width="533" height="317">
###MACRO: Return (CR) and Linefeed (LF) KM Test
~~~ VER: 1.0 2017-10-25 ~~~
####DOWNLOAD:
<a class="attachment" href="/uploads/default/original/2X/9/9d693898f57ee810ce95d4710a6ff596dc79c97c.kmmacros">Return (CR) and Linefeed (LF) KM Test.kmmacros</a> (3.7 KB)
**Note: This Macro was uploaded in a DISABLED state. You must enable before it can be triggered.**
---
<img src="/uploads/default/original/2X/f/f9657a52756e21967d773d0fe7b1bb19c057bb4e.png" width="473" height="1046">
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
### 2. AppleScript Test
<img src="/uploads/default/original/2X/b/b4dadb2eb25ce5a4c1c8760ed69e14de84cc1c6a.png" width="433" height="423">
###MACRO: Return (CR) and Linefeed (LF) Test from AppleScript
~~~ VER: 1.0 2017-10-25 ~~~
####DOWNLOAD:
<a class="attachment" href="/uploads/default/original/2X/8/846664e9aeb005f1fea8be67b294fe24526c7669.kmmacros">Return (CR) and Linefeed (LF) Test from AppleScript .kmmacros</a> (5.3 KB)
**Note: This Macro was uploaded in a DISABLED state. You must enable before it can be triggered.**
---
<img src="/uploads/default/original/2X/6/6502483568bba85a044f5180882c152e6b936c8f.png" width="473" height="1282">
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
### BBEdit
BBEdit 12.0.1 (400010) on macOS 10.11.6
Paste from both AppleScript and KM Clipboards
<img src="/uploads/default/original/2X/a/a92677db0e4b6201946b81823d6b4783f439060b.png" width="435" height="258">
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
### Clipboard Viewer.app
FROM:
```applescript
set sourceStr to "LF" & linefeed & "CR" & return & linefeed
set the clipboard to sourceStr
```
<img src="/uploads/default/original/2X/1/1f7199ca3ab469673007b3f9a35f41f70e5583b2.png" width="611" height="390">
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
### RegEx101.com Test
https://regex101.com/r/BCmTQi/2
RESULTS:
The `\r` does NOT match any character of text pasted from AppleScript or KM Clipboards.
CONCLUSIONS:
When text is pasted into this web page, this panel, it evidently converts all CR (ASCII 13) to LF (ASCII 10).
In the first example (with the unrecognized char) the CR does not work as line break, since the editor can’t handle two different kind of line breaks in one document.
The same in TextMate makes it a bit clearer:
The CR ist recognized as CR but does not “function” as a CR. The same in BBEdit: when you open the Character palette it is clearly identified as CR (Ascii 13), but BBEdit simply does not have a way to display it like TextMate.
But still interesting that the order decides if the CR gets converted into a LF or not. As you see in the screenshot, TextMate does not care about the order, it always preserves it as (non working) CR.
Yes, that’s what I tried to say in the last paragraph. (In contrary to TextMate, which preserves them.) Unless the CR comes first, then it does not convert it, for unknown reasons.
I think \n and \r are (almost) synonymous in BBEdit. Try to replace all \n by \r. You will still have linefeeds. It’s the document setting (at the bottom of the window) that decides over the type of linebreaks.
The line break setting (at the bottom of the window) decides which kind of line break is written to file.
In the editor (the open document) there are always LFs, even if the line breaks are set to “Legacy Mac (CR)”. However in the document on disk you’ll find CRs then.
To see what I mean:
Create a document consisting only of a line break.
Set the document to “Legacy Mac (CR)”.
Save it to disk.
Make a Hex dump of the document (File menu> Hex Dump Front Document)
You’ll get 0A (=LF)
Make a Hex dump of the file (File menu > Hex Dump File)
I think you're missing my point. I don't want to change all of the line breaks.
I may just want to change wherever I have a string like "some text\r" to something else.
KM can do this, so I guess I'll just have to do it there.
I think a text document with mixed line breaks is not a very sane document. So, honestly I’m very happy with the way BBEdit works in that respect.
Some years ago I had to work with Windows people and those produced very weird stuff sometimes. I highly appreciated that BBEdit made it so easy to “normalize” that.
But I understand that there might be needs to selectively treat CRs. Though I cannot think of a concrete example at the moment.
But yes, the system does mess around with them when it generates different flavors.
Do not use BBEdit as an example, it does many different things and has deep handling for line endings far exceeding (and different) to most applications.
Basically, you cannot rely on the line endings in the clipboard being anything in particular, and for most text flavors it is more likely to be CR than LF, even today.
Actually, you didn't say anything about applications. You said:
But what you said in correction makes more sense, and is what I was expecting.
IAC, I take your main point: Any searches need to check for either CR or LF (or even both). That's where the RegEx \R is so powerful. Unfortunately it isn't supported on all RegEx engines and/or all macOS.
So, to be safe, we could use something like (?:\r?\n|\r)
That should match any of these:
CRLF
LF
CR
OR, if you're looking for EOL, then maybe: (?:\r?\n|\r|$)
I'm sure you can improve on that.
But I still have one question: While using KM, what would be the advantage of using %Return%? Seems like LF (%LineFeed%) (shouldn't that be either "Linefeed" or "linefeed"?) is overall a better choice.
OK, let me answer my own question: Here's one good(?) reason to use %Return%: Microsoft Word 2011.
Word does NOT generate any kind of line break when you paste a LF. <shrug>
And it uses CR for line breaks when you copy text from Word.
I guess the unexpected should be expected with Word WRT the Clipboard.
Also, in Classic Mac OS there were nothing except applications. Besides, 99% of all clipboards copied on the Mac are copied from Applications.
No, CR on the clipboard is to be expected. That is perfectly normal for Mac. It seems the UTF8 flavor typically has line feeds, but other text flavors typically have CR, so it will depend which one is used in any given situation.
"line feed" is two words, tokens are Camel Case, and case sensitive, so it is %LineFeed%.
@peternlewis, OK, thanks for the info and the lively discussion.
Just so I'm clear, and maybe it will help others, here's a summary of my understanding.
2017-10-27 14:28 GMT-0500
• Correct use of \R per Peter's post.
Summary of Use and Handling of CR and LF
Submitted for your review, comment, and/or revision.
CR -- Carriage Return
ASCII Decimal: 13
Hex: 0D
RegEx: \r
KM Token: %Return%
.
LF -- Line Feed
ASCII Decimal: 10
Hex: 0A
RegEx: \n
KM Token: %LineFeed%
.
When using KM Actions with RegEx, it is best to allow for both CR and LF as newline characters. So a RegEx of something like this would be useful:
(?:\r?\n|\r|$)
\R will work macOS 10.11+
What is put on the Clipboard is dependent on the Application where the copy command is done. Newline character could be either CR or LF, but most likely CR. Again, we need to allow for either (see #1).
KM \n will match only LF in a Search and Search/Replace.
I thought I had seen cases where it matched CR, but I cannot reproduce that with Keyboard Maestro 8.0.3 (8.0.3) on macOS 10.11.6.
Peter, please confirm/correct.
BBEdit 12+ will match either CR or LF with a \n
When you paste source text to RegEx101.com, it converts all CRs to LFs.
I have reported this as a bug to the developer.
My Test Results and Macro
###MACRO:Â Â Â Return (CR) and Linefeed (LF) KM Test
2017-10-27 14:28 GMT-0500
