How Do I use RegEx (Grep) with AppleScript and Keyboard Maestro?

search
regex
files_folders
applescript

#1

This is not directly related to KM, except for the fact that my script is going to live in a KM macro eventually.

Basically I'm trying to quickly search for results in a text file using grep and Terminal. The command

grep -F http://myurl.com/whatever urldocument.txt

returns the line with the URL as needed. Frustratingly, using Applescript to get the result with

tell application "Terminal"
   do shell script "grep -F http://myurl.com/whatever urldocument.txt"
end tell

Breaks, because for some reason grep ignores (some?) commands with run through Applescript, certainly the -F part. My question is, why is this, and can I do anything about it?


#2

if it's run through KM, you could use the run shell script command, no?


#3

Hi, I hadn't thought to try it but just did so. Sadly, the bug, if that's what it is, still seems the same. Basically my script

grep -F https://www.jbox.com/category/apparel-cosplay /Users/pp/Documents/Dropbox/short\ URLs/jlist\ short\ URLs.txt

returns a single URL line when run manually in Terminal, but returns all lines, or hundreds of lines at least, when run through a run shell script. I thought it was ignoring the argument -F, but it might be something else as the same line without -F also returns just one line when run manually but all lines when done through KM or Applescript.


#4

You don't need to use a shell script for this. KM has very powerful and easy-to-use Actions that use Regular Expressions .

For your specific use case, this should work (or something similar):

Source Text File

image

Example Output

image


MACRO:   Extract Line from File that Matches Text [Example]

~~~ VER: 1.0    2019-02-05 ~~~

DOWNLOAD:

Extract Line from File that Matches Text [Example].kmmacros (4.1 KB)
Note: This Macro was uploaded in a DISABLED state. You must enable before it can be triggered.


image


Questions?


#5

Actually, it is directly related to KM. See my above solution.

@ppayne, I hope you don't mind that I have revised your topic title to better reflect the question you have asked.

FROM:
Grep + Applescript question

TO:
How Do I use RegEx (Grep) with AppleScript and Keyboard Maestro?

Providing a succinct, yet descriptive title about your problem/issue will greatly help you attract more experienced users to help solve your problem, and will help future readers find your question, and the solution.


#6

Hey Peter,

Don't do that!  :sunglasses:

A do shell script command has nothing whatsoever to do with the Terminal.

grep -F

Uses fixed strings rather than a regular expression. You don't need the -F switch, unless you specifically want to turn off RegEx.

In this case you haven't given the script an explicit target directory:

do shell script "grep -F http://myurl.com/whatever urldocument.txt"

Try running this to see where AppleScript thinks you are in the file system:

do shell script "ls"

(AppleScript doesn't remember where you set the working directory between do shell script statements.)

When running a shell script from AppleScript the best syntax is as follows:

set shCmdStr to "
cd /Users/chris/test_directory/
grep 'https://' 'Large_Doc_Test -- Very Large Doc -- 500,000 Lines.txt'
"
set stdOUT to do shell script shCmdStr

Shell scripts are finicky, and it's a good idea to eyeball the code before running it.

(This is also helpful for debugging).

You're generally better off for readability and safety reasons to use a change directory statement to move the working directory to your desired target directory and then performing your operation:

Execute a Shell Script.kmactions (689 B)

-Chris


#7

Thanks! I will study this well..


#8

Thank you very much! That is handy! I am once again in awe of KM. If I had to spend the rest of my life on a desert island with only one piece of software, I know which one I'd pick.


#9

Thanks, this is working pretty smoothly. There's one bit of voodoo though. The display text properly shows me the results I want, indicating that %Variable%Local__MatchedLine% has the correct value. But in the line below, where I try to put the value into the clipboard so I can work with a little more (parse out the URL I need), the value is reverted to its original value (NOT FOUND). Is there any reason why the value of %Variable%Local__MatchedLine% in the Display Text box wouldn't be the same as when other script commands try to work with the value? Speaking the value of %Variable%Local_MatchedLine% immediately after the display results command results in it's original value, not the results I need.


#10

the problem is that in your Set Clipboard you have a typo.
%Variable%Local__MatchedLine%
has TWO underscores. Yours has only one.


#11

Ack, thank you!


#12

Hey Peter,

Keep in mind that JM's macro will only find the first available line that fits the pattern.

It's pretty darn fast even on quite large files.

But – if you need to find multiple matches on a big file then Keyboard Maestro isn't the best tool for the job.

Grep only takes a couple of seconds to power through a 500,000 line file on my elderly MacBook Pro.

-Chris


#13

I know you have resolved this, but understanding the issue is also useful.

If grep is behaving differently in response to the -F switch when run from Keyboard Maestro/AppleScript than when run from the Terminal, then my guess is you have two variants of grep installed.

Try doing which grep in the Terminal and see if it is using a different grep instead of the normal /usr/bin/grep, Alternatively, it may be that there are some environment variables that are affecting the behaviour of grep.

Or just carry on with your existing solution, that also works…


#14

Where on Earth did you find a 500K line file????? LOL
Wait -- I know -- you created it using AppleScript! What else!

I've never had the need for anything close to 500K lines.