I'm trying to edit a text file and I want to know when I get to the end of it, either by recognizing the end of file mark or using the number of lines in the file. I haven't been able to figure out any way to do it. It may be obvious, I'm a newbie. Thanks for any help.
--Jim
If you are using KM, you should not need to be concerned about End of File.
How are you trying to edit?
It is best to upload your macro so we can better understand what you are trying to do.
Please read:
Tip: How Do I Get The Best Answer in the Shortest Time?
I want to reverse the order of the lines in a text file. My macro is incomplete and may be pretty awful. It's more of an exercise for me than essential. What I think I need is the Until condition, which I've done in other environments by EOF or linenumber = number of lines.
Reverse Lines2
A shell command for reversing lines is
tail -r file.txt
It could be incorporated into your macro.
3 Likes
@drdrang has given you the solution to reverse the lines in your file.
However, it needs some clarification.
@drdrang, could you please clarify?
I'm a bash novice, but my understanding is that the command you provided will only output the last 10 lines of the file to stdout.
I think the OP wants to reverse ALL lines in the file, output back to the file.
How would you do that?
@jimtparr, do you want to do further processing the the lines in the file, or just reverse the lines?
This makes a big difference in how the shell script is setup, and what additional KM Actions are needed.
For your reference, you can use the KM For Each action with a "lines collection" in ether a file or a KM variable. That will provide a KM Variable that contains each line as it loops through the text. You do not worry about the EOF -- KM handles that for you.
When you use the -r switch to reverse the order of the lines, tail outputs all of the lines by default, not just the last 10.
1 Like
OK, I assume you have tested that. I've read documentation on tail that says by default it only outputs the last 10 lines.
It’s in the man page and it works. Try it.
I responded by email, and it doesn't appear here as I thought it would.
On further consideration, I think it would be more trouble to get the filepath ahead of time than to do the copy to and paste from the keyboard. So I'll happily use my macro as it is, reversing the order of the lines in the system clipboard. In the process, I've learned more about KM, and I'm glad of that. Thanks for your help.
Actually, getting the FilePath is very easy in KM.
There are at last two methods:
OR
- For Each action with either a Folder Contents collection or a Finders Selection collection.
Here is a complete macro setup using the For Each method, but I have also provided a Prompt for File Action at the top, that is disabled. So you can easily reconfigure the macro if so desired.
Below is just an example written in response to your request. You will need to use as an example and/or change to meet your workflow automation needs.
Please let us know if it meets your needs.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MACRO: Reverse Lines in Finder Selected File [Example]
-~~~ VER: 1.0 2020-08-08 ~~~
Requires: KM 8.2.4+ macOS 10.11 (El Capitan)+
(Macro was written & tested using KM 9.0+ on macOS 10.14.5 (Mojave))
DOWNLOAD Macro File:
Reverse Lines in Finder Selected File [Example].kmmacros
Note: This Macro was uploaded in a DISABLED state. You must enable before it can be triggered.
ReleaseNotes
Author.@JMichaelTX
PURPOSE:
- Reverse the Lines in the Selected File
HOW TO USE
- Macro can be configured to either:
- Use One or More files selected in Finder (default)
- Changed to Prompt User for One File
- Using Default (as provided) Method:
- Select one or more files in Finder
- Trigger this Macro.
NOTICE: This macro/script is just an Example
- It is provided only for educational purposes, and may not be suitable for any specific purpose.
- It has had very limited testing.
- You need to test further before using in a production environment.
- It does not have extensive error checking/handling.
- It may not be complete. It is provided as an example to show you one approach to solving a problem.
REQUIRES:
- KM 8.0.2+
- But it can be written in KM 7.3.1+
- It is KM8 specific just because some of the Actions have changed to make things simpler, but equivalent Actions are available in KM 7.3.1.
.
- macOS 10.11.6 (El Capitan)
- KM 8 Requires Yosemite or later, so this macro will probably run on Yosemite, but I make no guarantees.
MACRO SETUP
- Carefully review the Release Notes and the Macro Actions
- Make sure you understand what the Macro will do.
- You are responsible for running the Macro, not me.
.
- Assign a Trigger to this maro.
- Move this macro to a Macro Group that is only Active when you need this Macro.
- ENABLE this Macro.
. - REVIEW/CHANGE THE FOLLOWING MACRO ACTIONS:
- ALL Actions that are shown in the magenta color
USE AT YOUR OWN RISK
- While I have given this limited testing, and to the best of my knowledge it will do no harm, I cannot guarantee it.
- If you have any doubts or questions:
- Ask first
- Turn on the KM Debugger from the KM Status Menu, and step through the macro, making sure you understand what it is doing with each Action.
The shell command works as you described, but I had two problem perhaps you could help us with:
- Output the results to the same file
- when I tried this:
tail -r "$KMVAR_Local__FilePath" | "$KMVAR_Local__FilePath" - I got this error:
- when I tried this:
/Users/UserName/Documents/TEST/KM Test/Reverse Lines Test copy.txt: Permission denied.
- Is there way to output to the source file?
- Reverse Fails if Last Line does not have a EOL character.
- So This as a source file:
- Results in this:
- So This as a source file:
I fix that by:
- Adding a LF at the end of the source file before processing.
- Adding a LF to the output of the
tailcommand:
*
Is there a better way to fix the line-ending issue?
I got it downloaded. I'm sure it's just what I wanted. I may not get a chance to try it right away, but I'll let you know when I do.
1 Like
tail -r "$KMVAR_Local__FilePath" | "$KMVAR_Local__FilePath"
There are a couple of problems with this command. First, the vertical bar is intended to send the output to another command for further processing. This type of redirection is called a pipe and is not what you’re looking for.
To simply redirect output to a file, use the greater than symbol, which looks like an arrow:
tail -r original.txt > reversed.txt
This might lead you to believe that you could change your command to
tail -r "$KMVAR_Local__FilePath" > "$KMVAR_Local__FilePath"
but that leads to the second problem: the > operator opens and truncates the file it points to before the preceding command is executed. Which means you’d be doing tail -r on an empty file. The rule of thumb is never to redirect the output of a command back to the original file.
I suggested tail -r because it was simple, and I thought @jimtparr wanted to do something with the reversed lines in Keyboard Maestro. To reverse the lines of a file in place, there’s a classic sed command:
sed -i '.orig' '1!G;h;$!d' file.txt
The -i '.orig' part creates a copy of the original unreversed file with a .orig extension. If you don’t need the original version of the file, use
sed -i '' '1!G;h;$!d' file.txt
The 1!G;h;$!d is a really tricky sed command that I won’t attempt to explain. Look here.
As for your second concern,
Reverse Fails if Last Line does not have a EOL character.
the failure is because tail -r is a very literal command. The definition of a “line” in most (all?) Unix commands includes the trailing newline character if present. So when tail -r reverses the lines, it prints each “line” in reverse order. With no newline at the end of “line 5,” nothing gets printed between it and “line 4.”
If you use the sed command to reverse the lines, this won’t happen. Like most Unix text editors, sed considers it a sin for a file to be without a trailing newline, so it adds one if it isn’t there.
There is a consistency to the classic Unix commands, but that consistency can lead to unexpected behavior in edge cases. Everyone gets burned by them now and then. I invite @tjluoma, who is far better with the shell than I am, to chime in and correct any mistakes in this answer.
3 Likes
I prefer a solution I can understand better. I don’t know any Unix, so I’d rather have a solution that doesn’t use it. With the two KM things suggested, I think I can do it without Unix. I don’t understand why my searching on “file” in the manual didn’t turn those up.
Thanks to both of you for your time and efforts. —Jim
Noli me vocare, ego te vocabo. Sit vis tecum. [Latin: Don't call me, I'll call you. May the Force be with you.]
Thanks for your detailed reply. It is very helpful. 
So, I guess after all is said and done, my KM solution of outputing the tail command to a KM Variable, and then writing that Variable to a file is good as any, and allows the user/editor of the KM Macro to easily output to a different file if he/she so chooses.
What would be the best way to test for a newline character at the end of the source file, and then to add one if not present?
I agree. Shell scripts can be very powerful, but also very dangerous if you don't know what you're doing. So, with a few "canned" solutions, I too try to avoid shell scripts.
I'm not sure what you mean. If you can be more specific, I'd be glad to update the KM Wiki.
However, the File Actions action in the Wiki seems to be very comprehensive, especially if you consider the "See Also" Actions at the bottom.
I hope you don't mind that I have revised your topic title to better reflect the question you have asked.
FROM:
Until End of File?
TO:
How Do I Know I’ve Reached the End of File (EOF) in a File I’m Editing with KM?
This will greatly help you attract more experienced users to help solve your problem, and will help future readers find your question, and the solution.
Short ways of adding a newline to the end of a file if, and only if, there isn’t already one there tend to rely on tricks. For example:
tail -c1 file.txt | read || printf "\n" >> file.txt
The tail -c1 gets the last character of the file. That’s then piped to read, which is normally used to gather a line of input and split it into words for later processing. But this one-liner cares only about read’s exit code, which is a success if the input ends with a newline and a failure if it doesn’t. The success or failure is then used by the short-circuit OR operator, || to determine whether the next command is executed. The final command, printf "\n" >> file.txt, outputs a newline and appends it to the file.
So, if the last character of the file is not a newline, read fails, the printf command is run, and a newline is appended to the file. If the last character of the file is a newline, read succeeds, the printf command is not run, and the file is left unchanged.
There’s also a sed command for this:
sed -i '' '$ a\' file.txt
The -i '' switch tells sed to edit the file in place and not to make a backup of the original file. The $ part of the command tells sed to act on the last line of the file, and the a\ part tells it to append nothing. As with the sed command in my previous answer, this works because of sed’s prejudice against lines that don’t end with a newline. When it appends “nothing” to the last line, it also tacks on a newline. Frankly, I don’t see why you’d want to use this if you didn’t like the earlier sed command.
Finally, we can use Perl:
perl -pe 's/$/\n/ if substr($_, -1) != "\n"' file.txt
This goes through every line in the file adds a newline if the last character of the line isn’t already a newline. This is pretty inefficient, as it looks at all the lines when all we really care about is the last one, but at least it doesn’t rely on tricks. How readable it is depends on how readable you think Perl is.
I haven't written enough Ruby to feel comfortable doing this in that language, although I suspect a Ruby solution would look a lot like the Perl solution. An equivalent Python script would probably be easier to read than any of these—even if you don't know Python—but it would also be distinctly longer.
What the “best” solution is depends on what you feel comfortable with and what you think you’ll remember when you go back to look at your code later.
Thanks again for your very detailed, and very helpful reply.
I didn't dislike the prior sed command -- I just already had a solution with tail and didn't see a reason to change.
But it looks like sed is the best overall solution for this task, since we won't have to worry whether or not the source file end with a newline character.
So, for the record, I think I'll redo my macro using your earlier sed:
sed -i '.orig' '1!G;h;$!d' file.txt
I like the idea of an automatic backup.
So I can add this shell script to my collection of "canned" solutions that I treat like a black box.
EDIT: sorry, one more question. Does it matter which command to use if the file is a very large file, say thousands or tens of thousands of lines?
My intuition about efficiency has led me astray so many times, I wouldn’t hazard a guess. I will only say I’d be surprised if any of them have trouble with large files.
UPDATE:
OK, I just ran some tests on a file with 100,000 lines, and the sed solution was so slow I stopped it before it could finish. What did I say about my intuition?
My suggestion for efficiency is to run this three-line shell command:
tail -c1 file.txt | read || printf "\n" >> file.txt
tail -r file.txt > file-reversed.txt
mv file-reversed.txt file.txt
These ran essentially instantaneously in my testing.
Obviously, you’ll want to make some Keyboard Maestro variables for the names of the original and temporary reversed files and use them instead of file.txt and file-reversed.txt.
FURTHER UPDATE:
The Perl solution is also fast and is only two lines:
perl -ne 'chomp;unshift @a, "$_\n";END{print @a}' file.txt > file-reversed.txt
mv file-reversed.txt file.txt