The Execute Shell Script action is behaving erratically. Specifically:
The action can fail even when the script itself runs.
An action that crashes when Failure Aborts Macro is turned on for that action can succeed when Failure Aborts Macro is turned off for that action.
Different commands producing exactly the same output behave differently. In the examples below, running stdout through tee or not made the difference between success and failure.
I'll demonstrate with three example shell scripts:
ls --- success
lsof +d /Users/jjh/tmp
2A With Failure Aborts MacroENABLED --- failure
2B With Failure Aborts MacroDISABLED --- success
lsof +d /Users/jjh/tmp | tee lsof.output --- success with Failure Aborts MacroENABLED
The only change from 2 to 3 is using tee to capture stdout and save it to a file. In this case the output Keyboard Maestro is coming from tee rather than from lsof.
Case 2B: Exactly the same as 2A, but with Failure Aborts Macro turned OFF. Even though KM thinks that the action failed, the code actually ran, and KM even captured the output and displayed it.
Thanks for that! I looked at stderr, which was empty. I'll be honest and say that my shell scripting is rusty, but my habit from back in the day is to equate no crash + empty stderr as a sign that all is well. I never wrote code that threw an error without some explanation as to why. I didn't look at exit status here, which I should have. That's the obvious thing for KM to check; next time I'll know.
I do find it puzzling that a standard tool like lsof routinely exits with a misleading status. I make a lot of use of command line tools, and always play with code in terminal before dropping it into KM. I'll add looking at exit status to preflight checks!
It was sort of happenstance that I ran into the problem at all. The final use looks like lsof +d ~/tmp | grep pdf | wc | awk '{print $1}', which gives a count of open PDF files in a directory. Fortunately awk seems to understand exit(0)! It's the best way that I've found to test whether things like web downloads are complete. Look at the number of open files up front, then wait until that number is back to where you started before moving on.
I only bumped into the problem because there is a lot of information there and I was thinking about parsing it. Lesson for the future: parse in the shell, dump the results in mktemp KM_XXXXXX, and return the filename. Then clean up when you are done. (Actually... let KM generate the tmp file, pass the name in a shell variable, let the shell return what it will, then let KM clean up after itself. That's probably a better way to avoid leaving a bunch of orphan tmp files laying around...)
But now I'm off into the weeds. Again, thank you very much for taking the time to figure out the error of my ways!
I should add... Over the last year-ish, KM has qualitatively changed my relationship with my digital world. I had never really seen a need to dive into AppleScript or AppleScript ObjC, and seldom made much use of JavaScript. Now I'm sure that JXA is in my future. I am enjoying a reason (excuse) to play in those waters while at the same time greatly improving my workflows.
In addition to @Nige_S’s information, you may want to look at the following discussion from June that also contains relevant information - and not just about grep:
While the KM Wiki page on Execute a Shell Script mentions stderr, I don't think that it mentions error status. It would be handy if there were a line early on that mentions that KM uses error status to judge success or failure, and that checking error status should up front in the process of troubleshooting problems.
It's the best way that I've found to test whether things like web downloads are complete. Look at the number of open files up front, then wait until that number is back to where you started before moving on.