Using installed shell tools from KM

Questions & Suggestions
1

Hi guys,

I'm a newbie at using the shell. I installed a tool called csvkit to help with some of my tasks. I'm using a command named csvstat. It works fine from the Terminal, but KM doesn't seem to understand the command. It returns this error:

/var/folders/w_/1s9d7ph56lx9dvbz_1q2bj9r0000gn/T/Keyboard-Maestro-Script-23194AB1-19B5-4F4F-890C-60843371A853: line 1: csvstat: command not found

I know the wiki gives the following disclaimer—maybe this is relevant?

Shells are executed in non-interactive mode … . As such none of your profile scripts (like ~/.profile) will be executed. This means that custom environment variable settings will not be applied, in particular any settings for your PATH environment variable, and any settings for tool-specific environment variables like PERL5LIB). You can set these by creating a variable with “ENV_” at the start (eg ENV_PATH or ENV_PERL5LIB) and settings its value appropriately.

TIA

2

Not sure, but you could try specifying the complete path to csvstat, until someone else has a better solution.

Like “/Applications/csvstat.app”, although it it actually is in “Applications”, then this isn’t likely to be the issue.

1 Like
3

Thanks, Dan. It’s not an app, but an installed command-line tool. Do you know where that would be installed?

4

you can use "which" or "whereis" in terminal to locate shell files. Unfortunately I don't find a file of thet name OMM.

john-iMac:~ john$ which csvstat
john-iMac:~ john$ whereis csvstat
john-iMac:~ john$ which ls
/bin/ls
john-iMac:~ john$ which textutil
/usr/bin/textutil

'/usr/bin/' would be a good place to start.

2 Likes
5

That file is available on github, http://csvkit.readthedocs.io/en/latest/scripts/csvstat.html

1 Like
6

Maybe one of the following will help:

  1. How Scripts Are Processed
  2. Turn CSS into Stylus - #12 by ccstone
1 Like
7

Hey Jack,

If you're going to fool with 3rd party stuff please always provide the details of where you got it from (and if relevant – how you installed it).

This is all Python stuff and looks like it has dependencies, so how you installed it is very relevant.

-Chris

1 Like
8

If you’re using the bash shell, you can force it to run from a login shell by using a shell script like:

bash -lc '/Users/myname/Desktop/myscript.sh'

That will keep your $PATH variables up to date so your installed stuff ought to work.

3 Likes
9

Thanks all.

I think @ccstone is on the right track asking how I installed it, because that answers the questions of where it lives.

I installed it from instructions here using the command:

sudo pip install --ignore-installed csvkit

By the way, the name of the tool is csvkit. The name of the command is csvstat.

I can find from spotlight that I have a csvstat.py at

Library > 
Python > 
2.7 > 
site-packages > 
csvkit > 
utilities > 
csvstat.py

Oh also, the files are available here.

10

Hey Jack,

I had found the files on GitHub, but the relevant pages didn't give me a satisfactorily clear method of installation – particularly since pip is not installed by default on OSX.

So I had to figure out how to install pip, before I could continue. With that accomplished it was easy to install csvkit, and with csvkit installed it became easy to test.

This works for me:

-Chris

2 Likes
11

@ccstone Wow, I’m impressed! Thanks so much for doing that extra sleuthing, and sorry I wasn’t clearer before. Yes, that’s solved it.

How did you end up finding csvkit after you had installed pip? With whereis?

Also, can you explain the #!/usr/bin/env bash -l line? It looks like you are simply telling bash to run with my login environment—in which case it doesn’t matter where csvkit is—right?

12

Hey Jack,

whereis checks the standard binary directories for the specified programs.

which searches the path for each executable file that would be run had the command actually been invoked. (Very important when searching for user-installed items.)

which csvstat

Precisely.

Right.

Since your log-in environment alters the stock $PATH to include your Python stuff.

-Chris

1 Like
13

Thanks! This is extremely helpful. So if an installed tool is always accessible from my login environment, then I can get to it in KM with #!/usr/bin/env bash -l, right?

14

Hey Jack,

If you're using Bash then it should.

Personally I don't like to pollute my Keyboard Maestro environment with stuff from my log-in environment, because of the potential for unintended side-effects – so I only use that sort of thing when it's needed.

When I don't need to pick up environment variables, aliases, or functions from my custom log-in files but do need items outside of the default $PATH structure I do one of a couple of things:

A) You can set Keyboard Maestro's environment variables the Editor's variable preferences:

Set a variable named:

ENV_PATH

To:

/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

That covers me for most things, but I haven't started fooling with Python (yet) either.

B) I alter the $PATH variable in the first line of my shell script.

#!/usr/bin/env bash
export PATH=/opt/local/bin:/opt/local/sbin:/usr/local/bin:$PATH;

This produces the same result as setting the environment variable above but is transient.

-Chris

1 Like
15

As @ccstone has alluded to, its not generally a good idea to use bash -l. It might get you working, but long term in may also be unreliable. There is a reason that your login scripts and such are not read automatically when you run scripts, because they change the environment in unpredictable ways. For example, you might think later it is a good idea to have your bash login print out a fortune or the date or some status information (maybe your harddisk free space for example). All good ideas, but then your script running with bash -l now gets strange extra output that is not expected.

So while using bash -l to quickly check that things can be made to work might be ok, better is then to avoid it and configure just the required environment settings appropriately.

1 Like
16

Thanks, @peternlewis and @ccstone. This is good to know. I see why I might not want all the (potential) cruft of my bash login to transfer to my KM scripts.

Could either of you explain a little bit more what it means to change the environment variables? It looks like I’m telling the shell, “please pick up any installations I might have in the following folders….” Is that correct?

And am I correct in thinking that my having used python for the csvkit tool complicates the matter because we’re not sure where that was installed?

17

Open the Terminal, and type “env” return. It will show you what the environment looks like in the Terminal, with your login options set. There will be options in there that affect the running of commands, such as the PATH environment variable which controls where the system will look for commands that don’t have a full path. There may be other environment variables that affect the tools that you use, for example the PERL5LIB environment variable controls how the perl tool will search for libraries.

You can set all the applicable environment variables in your Execute Script action, before executing the command you want to perform, or alternatively, you can set them in the Keyboard Maestro variables by adding “ENV_” to the front of the name, for example the Keyboard Maestro ENV_PATH will be used to set the PATH environment variable before executing scripts.

1 Like
18

How do you just append to the path, instead of resetting the entire thing?

19

You cannot do that by setting a Keyboard Maestro variable - it is all or nothing.

You can do that within the Execute Shell Script action at the start by setting the PATH appropriately, and using the PATH as part of the value you set (in csh, it is setenv PATH /opt/local/bin:/opt/local/sbin:$PATH, I’m not sure about bash scripts).

1 Like
20 
export PATH=/opt/local/bin:/opt/local/sbin:/usr/local/bin:$PATH;
2 Likes