What is Best Method to Prepend A Few Lines of Text to a Large File?

I know KM has a nice tool to append: Append Text to File, but I don't see anything to prepend.

I know there are number of shell script and AppleScript gurus in our members, so I'm hoping someone has already solved this problem:

What is Best Method to Prepend A Few Lines of Text to a Large File?

By large I mean 2,000 - 3,000 lines. I know, that's not really large, but I need this to work very fast because I'm using this in a dynamically created file in a Prompt With List action.

TIA.

Well - @JMichaelTX, I waited until a guru responded to your request. Since they didn't, I'm fine with embarrassing myself to give you my solution to shred up. I'm pretty sure my method won't suffice, for your need, but it works for my prepending needs. My hope is maybe it will trigger a faster/better solution from you or someone else. I copied a whole AppleScript PDF manual (771 pages), to test, on my 2014 iMac and it took about 10 seconds.

Anyway, enough chatter.

Here is the macro but don't embarrass me too much:

Write to Test file (Prepend).kmmacros (6.7 KB)

KC

3 Likes

I think you've got it about right. The general approach I've always used is:

  1. Save new text to a variable.

  2. Open the original file for reading and writing.

  3. Read the file into a variable.

  4. Write the new text followed by the original file text to the original file.

3 Likes

Here's another way.

Search and Replace Action (v9.0.5)

Search and Replace.kmactions (519 B)

6 Likes

@thoffman666 - I like that! :sunglasses:

Many thanks. It seemed like an ideal use for \A.

For those who don't already know, a search and replace of \Z can be used to append.

1 Like

Unfortunately there isn't really a good way to prepend information to a file.

All of the "solutions" that I am aware of are basically variants of this:

  1. Take the existing file's data and store it (either in RAM or in a temp file)

  2. Add the new information to the file

  3. Append the old information to the new file

That being said, I bet that would probably be fast enough for your purposes.

Here's a simple prepend.sh shell script which will take whatever input it is given and prepend it to a file, which is define as FILE=

(You can/should edit that line to the appropriate file on your system.)

#!/usr/bin/env zsh -f
# Purpose: add new information to the top of a file
#
# From:	Timothy J. Luoma
# Mail:	luomat at gmail dot com
# Date:	2020-07-10

	## Change this to wherever your file is
FILE="$HOME/Documents/SomeLargeFile.txt"

## Shouldn't need to change anything below this line ##

NAME="$0:t:r"

	# a useful default for macOS
PATH='/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin'

	# if the script is given no arguments, don't do anything
if [[ "$#" -gt "0" ]]
then

		# does the file exist and have a size greater than zero bytes?
	if [[ -s "$FILE" ]]
	then

			# if we get here, the answer is yes

			# this lets us use `$EPOCHSECONDS` as a variable
		zmodload zsh/datetime

			# a random temporary filename
		TEMPFILE="${TMPDIR-/tmp/}${NAME}.${EPOCHSECONDS}.$$.${RANDOM}.txt"

			# there's no way that file should exist, but let's remove it if it does
		rm -f "$TEMPFILE"

			# move the existing file to the temporary file
		mv -f "$FILE" "$TEMPFILE"

			# now we take whatever input we are given and save it to the file
		echo "$@" >>| "$FILE"

		## OR, remove the previous line and use this with Keyboard Maestro shell variable
		#
		# echo "$KMVAR_Whatever" >>| "$FILE"

			# now we append all of the information which was previously in the file
			# and delete the temp file
		cat "$TEMPFILE" >>| "$FILE" && rm -f "$TEMPFILE"


	else

			# if we get here, the file didn't exist before, so we just create it

		echo "$@" >>| "$FILE"

	fi
fi

exit 0
#
#EOF

2 Likes

Wow! What GREAT bunch of people we have in this forum, with expertise not only in KM but in many other related tools!

I do not dislike any of your solutions. Because you guys have provided such a good selection of solutions, I'm going to spend some time this weekend analyzing and running some tests on them to see what works best for my use case.

Thanks again!

4 Likes

Perhaps I'm missing something but why not just use cat.

dump your two lines to file_a cat file_a and your file together.

Could you please provide a complete, detailed, example for those of us that are bash-challenged?

# Create sample 'main' file
echo "line 1" > /tmp/main_file.txt
echo "line 2" >> /tmp/main_file.txt
echo "line 3" >> /tmp/main_file.txt
echo "line 4" >> /tmp/main_file.txt

# create file to 'prepend'
echo "line 0" > /tmp/file_to_prepend.txt

# using 'cat' to concatenate the two files
cat /tmp/file_to_prepend.txt /tmp/main_file.txt > /tmp/new_file.txt

# move the new file back over the original file
mv /tmp/new_file.txt /tmp/main_file.txt
2 Likes

If the file size is small enough, then just read it in to a variable, and save it back.

But to avoid it going in to a variable, the cat suggestion is probably the best.

Expanding on that, this looks like a nice solution:

cat - ~/File.txt >~/Temp.txt
mv ~/Temp.txt ~/File.txt

image

\z is generally better for this - \Z actually matches before the trailing line ending at the end of the file, where \z matches at the very end, mirroring \A.

3 Likes

Absolutely the better use of stdin on cat there ^^^.

Just wanted to chip in and share my version of solving this problem. I took a slightly different approach but ran into strange behavior with $KMVAR_SystemClipboard in shell scripts.

Most elegant solution IMO is @thoffman666's regex approach, but I want the timestamp in all caps with specific formatting while preserving KM's clipboard history with minimal actions. Different purpose, similar problem.

In my macro, I copy text and immediately try to use it via $KMVAR_SystemClipboard in an Execute Shell Script action. Even with a 5-second pause after Copy, the variable evaluates wrong in the shell script. However, if I add a "Set Variable to Clipboard" action immediately after Copy, then reference that variable in the shell script, it works perfectly.

It seems like $KMVAR_SystemClipboard doesn't reliably capture the system clipboard state when evaluated inside shell scripts, even though the clipboard itself contains the copied text (present in Clipboard History Switcher). The workaround is using "Set Variable to Clipboard" to cache it first.

Is this expected behavior, or should $KMVAR_SystemClipboard work directly in shell scripts after a Copy action?
Am I missing something obvious about how 'Set Variable to Clipboard' should work?

Prepend | Textbox to typing.txt [KM].kmmacros (5.6 KB)
—————————————-

This is what the output looks like:

$KMVAR_SystemClipboard will evaluate the KM variable SystemClipboard, not to the contents of the current macOS System Clipboard. So yes, you have to set that variable after the Copy.

But a probable better method is to access the System Clipboard directly in your shell script, using the pbpaste command -- replace "$KMVAR_SystemClipboard" with `"$(pbpaste)":

file="/Users/nigel/Desktop/typing.txt"
datetime=$(date "+%a, %b %d, %Y [%H:%M]" | tr '[:lower:]' '[:upper:]')
printf "%s\n\n%s\n---\n\n" "$datetime" "$(pbpaste)" | cat - "$file" > "$file.tmp" && mv "$file.tmp" "$file"
1 Like

For anyone who reads this post regarding prepending text to a file, and who may be interested to have a look at the macro below, which always prepends (Title, URL, Clipboard) to a specific file and creates a kind of a log.
(With keyboard keys (arrow up), but this is better than nothing.)

Send4 → Save this post- section to specific file.kmmacros (6.0 KB)

Thank you @Nige_S for helping me better understand this and for catching that issue. You're like Guilfoyle from Silicon Valley with that one-liner fix :wink:
I fixed it, tested it with over 2000 characters, and it works smooth as hell. Below is the updated solution and the macro.

Prepend | Textbox to _typing.txt v 1.2.kmmacros (5.7 KB)