Using a file path variable when executing a shell script

Hi,

I'm experimenting with executing a Python script file via KM and, after reading a few forum posts, I thought I could use a KM variable to store the path to the Python file and use that in an Execute Shell Script action. I've created a simple action to test this:

I have the py_icoudpath variable set to /Users/me/Library/Mobile\ Documents/com~apple~CloudDocs/Coding/Python

But when I run the macro I get an error:

Execute a Shell Script failed with script error: ls: /Users/me/Library/Mobile\ Documents/com~apple~CloudDocs/Coding/Python: No such file or directory.

To protect against potential typos, I ran ls /Users/me/Library/Mobile\ Documents/com~apple~CloudDocs/Coding/Python in Terminal and got the expected output of the folder's contents.

Please could someone tell me where I am going wrong?

Try removing the backslash from your pathname. Works for me.

So it's ...Mobile Documents...

1 Like

That works perfectly, thank you for the (nice and simple) solution!

1 Like

You'll run into this a lot, so it might be worth knowing why @tiffle's solution worked...

In the shell, a space separates the arguments to a command. So when there's a space in a file path you have to to the shell "this is a string with a space in it, not two separate arguments".

You can do that by quoting the path, which is what you did in your "Execute Shell Script" action. You can do that by "escaping" the space character with a preceding \, which is what you did in Terminal. But when you do both you are including the literal \ in the file path, which is why it doesn't work.

2 Likes

Thank you for the explanation @Nige_S, it's definitely helpful to understand the reason why @tiffle's solution worked :+1:

I'm curious though: if I add the escaping backslash back into the variable and remove the quotes from the action, the escape character doesn't seem to work as the error shows the path truncated at the space: /Users/me/Library/Mobile\: No such file or directory. Why would this be?

It's all down to the weird and wonderful ways of variable expansion and the shell. Doing the same also fails in Terminal. Compare the following:

myVar=/Users/me/Library/Mobile\ Documents;ls $myVar
myVar=/Users/me/Library/Mobile\ Documents;ls "$myVar"

You're setting the variable to a single string (the\ escaping the space) but it isn't a literal \, so the ls command gets the space-separated version unless you double-quote it. Things get even more complicated because, in the shell and in KM text boxes, \ is a special character and to include it as a literal you have to escape it with a \...

Not long after that, my head explodes...

Simplest solution is to not use \ but to always double-quote your path variables (the super-cautious also use brackets so the shell is in no doubt of the variable name):

ls "${KMVAR_myPath}"

Lots of good stuff about using KM variables in the shell and other scripts on the Wiki.

1 Like

too funny...

Sorry for the delay, I was scooping my brains back in after reading that as well! :smile:

Thank you very much for the extra detail. If I understand correctly, the path in the variable is a string, and the shell splits the string variable, at its spaces, into an array of arguments which it then passes to the command (ls, in this case). When this happens, the backslash in the string is treated as an actual backslash character (and not an escape character) so the space after it remains, causing a split at that point. This means that ls has a 1st / path argument of /Users/me/Library/Mobile\ and Documents as a second argument? Is that correct? If not and explaining again would cause you further head-detonations, please be assured that I'm happy to remain ignorant! :laughing:

I'm also a bit confused that both of your examples work fine in Terminal for me, but I can feel my brain starting to leak out of my ears again so time for a lie down I think!

Because your default shell in Terminal is zsh. Compare the following (echo $0 prints the name of the current shell):

user-166% echo $0
zsh
user-166% myVar=/Users/me/Library/Mobile\ Documents/com~apple~CloudDocs;ls $myVar
ls: /Users/me/Library/Mobile Documents/com~apple~CloudDocs: No such file or directory
user-166% bash
bash-3.2$ echo $0
bash
bash-3.2$ myVar=/Users/me/Library/Mobile\ Documents/com~apple~CloudDocs;ls $myVar
ls: /Users/me/Library/Mobile: No such file or directory
ls: Documents/com~apple~CloudDocs: No such file or directory
bash-3.2$ 

You can see that in zsh only one "No such file" error is reported -- no split on the space in the path -- while in bash there are two errors -- path is split on the space. To make things even more confusing, unless you've changed things the KM "Execute a Shell Script" action uses sh, which you can think of as the "lowest common denominator"...

Yet another reason to use quoted paths -- portability!

Nerdy bit!

To confuse things even more, there's no sh in macOS -- it's actually bash run in sh-compatability mode!

Whenever you've a problem in an "Execute a Shell Script" action, start by checking that what you are doing works in sh and that your environment variables are what you expect them to be (KM "Shell Script" actions have their own, not necessarily the same as you have in Terminal). And, of course, that gives rise to another solution -- instead of making your script work in sh, make the shell suit your script:

Shell Test.kmmacros (2.6 KB)

2 Likes

Wow, if I thought my mind was blown before... :grin:

Thank you again, that's a fantastic explanation and I can actually make sense of it, which is testament to its quality. It's also useful to have this understanding beyond its applicability to KM.

For all future shell scripts, I'll use the zsh shebang and make sure to quote (and probably bracket) my paths!

To make sure I'm not giving you the wrong idea...

You bracket the variable name to make sure the shell doesn't misinterpret it. The usual time you'll need it is in string interpolation, to show the shell where the variable name stops and the string starts:

sh-3.2$ myVar="scone"
sh-3.2$ echo "Have some $myVars, my dear"
Have some , my dear
sh-3.2$ echo "Have some ${myVar}s, my dear"
Have some scones, my dear
1 Like

Sorry, my mistake, I should have said "variables", not "paths". Thanks for the clarification and for the illustration (I wouldn't necessarily have thought of that potential problem or have known how to get around it if it occurred).