(Old) Issue with Shell, still can't figure out why this isn't working

alltiagocom · May 26, 2024, 8:57am

I installed ffmpeg and every time I have to use it inside KM I always have to add this to the beginning of the script:
/opt/anaconda3/bin/

I already have this in my variables:

Initially, from what I was told, this would fix the issue and I would just have to add the script, like this, for example:

ffmpeg -f avfoundation -i ":3" -t 3 /Users/dannywyatt/Library/Mobile\ Documents/iCloud~md~obsidian/Documents/Tiago/System/Attachment\ Files/audio/audioaudiocapture.mp3

But to make it run in KM, I have to have this:

/opt/anaconda3/bin/ffmpeg -f avfoundation -i ":3" -t 3 /Users/dannywyatt/Library/Mobile\ Documents/iCloud~md~obsidian/Documents/Tiago/System/Attachment\ Files/audio/audioaudiocapture.mp3

So this is the first issue, if someone can help me with this?
I'm not an expert, but this seems to be the issue with certain workflows, for example if I want to use different variables to create a final script and then merge everything into a single variable and just add $KMVAR_Local__myVarHere to the Execute Shell action, it doesn't work, even though the whole script is working when I paste it directly to the Execute Shell action.

So, this does NOT work:

Execute a Shell Script failed with script error: text-script: line 1: ffmpeg: command not found

This DOES work:

This does NOT work:

2024-05-26 09:50:43 Action 15907120 failed: Execute a Shell Script failed with script error: ffmpeg version 4.3.2 Copyright (c) 2000-2021 the FFmpeg developers
  built with clang version 11.1.0
  configuration: --prefix=/opt/anaconda3 --cc=x86_64-apple-darwin13.4.0-clang --disable-doc --disable-openssl --enable-avresample --enable-gnutls --enable-gpl --enable-hardcoded-tables --enable-libfreetype --enable-libopenh264 --enable-libx264 --enable-pic --enable-pthreads --enable-shared --enable-static --enable-version3 --enable-zlib --enable-libmp3lame --pkg-config=/Users/runner/miniforge3/conda-bld/ffmpeg_1627813741069/_build_env/bin/pkg-config
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
[AVFoundation indev @ 0x7fb94c704280] Video device not found
":3": Input/output error

If I save this code to an .sh file and try to read it:

Execute a Shell Script failed with script error: /bin/sh: /Users/dannywyatt/My Files/AppleScript, Apps, Automator, Scripts, Droplets, etc/Shell Scripts/audiocapture.sh: Permission denied

I get this error reading from the .sh file whether I add /opt/anaconda3/bin/ to the beginning of the script or not.

ComplexPoint · May 26, 2024, 10:30am

The error message suggests a separate issue – distinct from the earlier problem of finding the ffmpeg executable, which I think you have now resolved.

Something to do with:

etc/Shell Scripts/audiocapture.sh

?

The space between Shell and Scripts ?
Permissions settings audiocapture.sh ?
Permissions or path for access to /etc/ ?

etc.

griffman · May 26, 2024, 1:10pm

As for the shell script issue with ffmpeg, this just came up in another thread, where it's really tricky to get the quotes correct when passing things in a var to ffmpeg due to embedded quotes. See @Nige_S' workaround in that thread:

(I posted a stupid silly solution which also works, which is to just type the command into Terminal via KM. Not the right way, but I was just trying to find any way to get it working :)).

-rob.

alltiagocom · May 26, 2024, 4:24pm

Not really... the issue is that it seems that other people are able to use certain commands without having to add the path, whereas I can't. I wish I was able to just add the command without always adding that extra info. Now I have a saved action with that already in there, just in case, but it would be good to know what's causing that?
Initially it seemed that all I needed was that global variable (ENV_PATH), but it seems that it's not doing anything...

I was able to make the script and macro work and here's what I found (@griffman I will check your shared link as well):

We know that a path without quotes in Terminal, for example, needs to have the spaces escaped, right? So in my mind I thought "ok, so if I want to use the path in the Execute Shell action, the variable needs to have the escaped spaces", so that's what I did. But then when I want to include the variable in the Shell action, I have to use quotes like this "$KMVAR_Local__myVar", otherwise it won't read the variable, so it seems that this automatically makes the path work without the need to escape the spaces (similar to when we use a path with quotes in Terminal), so when it tries to read a path from a variable that has escaped spaces, it's using the \ as part of the path:
my/path/to\ a\ folder will be interpreted by the script as the real path, including the backslashs instead of"
my/path/to a folder

That seems to be the issue, as I was able to make the script and macro work after I understood this. I even took a note on my KM notes in case I forget about it.

So yeah, now I was able to make the macro work.
I still wish the issue with the ffmpeg path wasn't there though.

Nige_S · May 26, 2024, 9:21pm

This may not be helped by using conda to install packages while most people use brew. While you should be able to make it work you may have limited your available Forum guinea pigs... Is there are reason you're using conda?

What error do you get if you don't run that? (I'm wondering if you might have a brew-installed version as well, and things are getting confused.) But you could try adding /opt/anaconda3: to the beginning of ENV_PATH so your conda installs are preferred over any others.

No, you don't have to use quotes. You wrap the variable in quotes so you don't have to escape the spaces in the variable, just like you wrap the path in quotes when you type it into Terminal. And since, as Mac users, we often use spaces in our file and folder names -- it's a good idea to always quote variables that contain paths.

Paths, shell variables, KM variables, and how they all work together is tricky. But keep the Wiki page to hand and you'll be OK.

alltiagocom · August 27, 2024, 9:29am

I need to go back to this topic, because it always creates this anxiety when I have to use something new with shell...
This time, it's ImageMagick that isn't working.
I have this script in a shell action:

magick "$KMVAR_Local__VarName" -sampling-factor 4:2:0 -strip -quality 85 -interlace JPEG -colorspace RGB "$KMVAR_Local__parentPath"-optimized.jpg

And I get this:

Execute a Shell Script failed with script error: text-script: line 1: magick: command not found

I tried adding the full path to the magick file, but it didn't work:

/usr/local/ImageMagick-7.0.10/bin/magick "$KMVAR_Local__VarName" -sampling-factor 4:2:0 -strip -quality 85 -interlace JPEG -colorspace RGB "$KMVAR_Local__parentPath"-optimized.jpg

2024-08-27 10:24:04 Action 16033870 failed: Execute a Shell Script failed with script error: dyld: Library not loaded: /ImageMagick-7.0.10/lib/libMagickCore-7.Q16HDRI.8.dylib
Referenced from: /usr/local/ImageMagick-7.0.10/bin/magick
Reason: image not found
text-script: line 1: 61377 Abort trap: 6 /usr/local/ImageMagick-7.0.10/bin/magick "$KMVAR_Local__VarName" -sampling-factor 4:2:0 -strip -quality 85 -interlace JPEG -colorspace RGB "$KMVAR_Local__parentPath"-optimized.jpg
2024-08-27 10:24:04 Execute a Shell Script failed with script error: dyld: Library not loaded: /ImageMagick-7.0.10/lib/libMagickCore-7.Q16HDRI.8.dylib
Referenced from: /usr/local/ImageMagick-7.0.10/bin/magick
Reason: image not found
text-script: line 1: 61377 Abort trap: 6 /usr/local/ImageMagick-7.0.10/bin/magick "$KMVAR_Local__VarName" -sampling-factor 4:2:0 -strip -quality 85 -interlace JPEG -colorspace RGB "$KMVAR_Local__parentPath"-optimized.jpg. Macro “]]]” cancelled (while executing Execute Shell Script).

This is driving me crazy... why is most people able to make this work and not me?
I still have to have that extra path for ffmpeg, no matter what.

My ENV_PATH is
/usr/local/bin:/opt:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin

I don't know if this is relevant, but I have both .zshrc and .bash_profile files in my home folder, and here's what they contain. I'm mentioning this, because ChatGPT mentioned both today when it was helping me install ImageMagick:

.zshrc

# Homebrew path
export PATH="/usr/local/Homebrew/bin:$PATH"

# Homebrew shell environment
eval "$(/usr/local/Homebrew/bin/brew shellenv)"

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/opt/anaconda3/bin/conda' 'shell.zsh' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/opt/anaconda3/etc/profile.d/conda.sh" ]; then
        . "/opt/anaconda3/etc/profile.d/conda.sh"
    else
        export PATH="/opt/anaconda3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<

# ImageMagick path
export MAGICK_HOME="/usr/local/ImageMagick-7.0.10"
export PATH="$MAGICK_HOME/bin:$PATH"
export DYLD_LIBRARY_PATH="$MAGICK_HOME/lib/"

.bash_profile


# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/opt/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/opt/anaconda3/etc/profile.d/conda.sh" ]; then
        . "/opt/anaconda3/etc/profile.d/conda.sh"
    else
        export PATH="/opt/anaconda3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<

I would really see if I can fix this issue once and for all, if anyone can help me?

alltiagocom · August 27, 2024, 9:35am

For some reason I never got to reply to your message. I must have skipped the notification.
When I tried to install ffmpeg, that was the option that was offered to me. I have no idea how packages and managers work, so I just did what I saw online.

I believe I have brew installed, but since I'm still on Catalina and it doesn't support brew, I always get errors when I try to install a new package

brew folder

I remember at one point in the past where I tried using a variable without quotes, the original path or whatever I was using, didn't have spaces, but it didn't work. Only worked with quotes, hence my statement. I can't recall what it was, though.

Nige_S · August 27, 2024, 10:22am

In your .zshrc file, this:

        export PATH="/opt/anaconda3/bin:$PATH"

and these:

# ImageMagick path
export MAGICK_HOME="/usr/local/ImageMagick-7.0.10"
export PATH="$MAGICK_HOME/bin:$PATH"
export DYLD_LIBRARY_PATH="$MAGICK_HOME/lib/"

...are setting your Terminal session's PATH and the DYLD_LIBRARY_PATH environment variable -- that isn't happening in your "Execute Shell Script" action.

For a test, try including them in the action's script:

#!/bin/bash
export PATH="/opt/anaconda3/bin:$PATH"
export MAGICK_HOME="/usr/local/ImageMagick-7.0.10"
export PATH="$MAGICK_HOME/bin:$PATH"
export DYLD_LIBRARY_PATH="$MAGICK_HOME/lib/"

magick "$KMVAR_Local__VarName" -sampling-factor 4:2:0 -strip -quality 85 -interlace JPEG -colorspace RGB "$KMVAR_Local__parentPath"-optimized.jpg

...and see what happens.

alltiagocom · August 27, 2024, 10:29am

That worked! Thanks!

I guess that for ImageMagick I don't need the first line:
export PATH="/opt/anaconda3/bin:$PATH"

So is there a way to make it so when I use the Execute Shell action I don't have to think about all of these "tricks" and just use the command the same way as I use it in Terminal?

I can save this to use with ImageMagick, but if in the future I add another package, then I have to keep asking for help. Is there something I can do?

alltiagocom · August 27, 2024, 10:32am

By the way, for the ffmpeg command I always have it set like this as a favorite action:

Nige_S · August 27, 2024, 10:38am

That's the one I would add to KM's ENV_PATH -- if you're installing more packages with Anaconda you may need it. It isn't the individual packages that are the problem -- as you've seen with ffmpeg you can "target" them on the command line -- but if those packages reference other packages that Anaconda has installed then they'll need to know where to look.

I'd keep the ImageMagick ones out though, and set up a favourite "magick" action that included those 3 lines at the top of the script. A clean PATH is a happy PATH, and the less that's in there the less you have to sort through when things go pants!

alltiagocom · August 27, 2024, 10:59am

Here?
/usr/local/bin:/opt:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin

Any suggestion on how that path "should" look like in terms of the order of the paths, including anaconda?

As I mentioned, I'm not familiar with how the packages work and all that, and when I see something online they always mention brew anyway. Now the issue, as I also mentioned, is that I can't seem to rely on brew to install anything, because it always stalls at one point. Today I had to install ImageMagick "manually".

I installed other packages via pip (I guess that's another package manager or something?). At least the instructions that I was given, mentioned installing via pip.

For me this is all a big mess and it annoys me when I don't understand these things... along with the issue of brew not working on my macOS.

So something like this

For ffmpeg I don't mind it like this, as I have a favorite action anyway

But if I can set the ENV_PATH properly and avoid having the path /opt/anaconda3/bin in the action, why not?

Nige_S · August 27, 2024, 11:30am

Yes. PATH is checked in order, stopping on first match. So when you use ls the shell goes

Can something named ls be found under /usr/local/bin? No. So...
Can something named ls be found under /opt? No. So...
Can something named ls be found under /usr/bin? No. So...
Can something named ls be found under /bin? Yes! Run it.

So you generally want your "special paths" at the front of the list so you can override any defaults. In this case:

/opt/anaconda3/bin:/usr/local/bin:/opt:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin

...and remember that that would mean that if you install the same-named utility with both Anaconda and Homebrew then the Anaconda one would be used.

Assuming MAGICKHERE is placeholder for your magick comand, then yes. If it is then I'd avoid accidents by making it a comment instead:

#!/bin/bash
export MAGICK_HOME="/usr/local/ImageMagick-7.0.10"
export PATH="$MAGICK_HOME/bin:$PATH"
export DYLD_LIBRARY_PATH="$MAGICK_HOME/lib/"

# magick command goes here

Bang on. And it also would mean that when you updated to anaconda4 or got an Apple Silicon machine and started using brew's ImageMagick instead you would have to go through and change all your actions -- you'd just update ENV_PATH.

Similarly, do watch out for any update to ImageMagick since we have hardcoded the version number into the action...

alltiagocom · August 27, 2024, 11:37am

Nige_S:

Yes. PATH is checked in order, stopping on first match. So when you use ls the shell goes

Can something named ls be found under /usr/local/bin? No. So...

Can something named ls be found under /opt? No. So...

Can something named ls be found under /usr/bin? No. So...

Can something named ls be found under /bin? Yes! Run it.

So you generally want your "special paths" at the front of the list so you can override any defaults. In this case:
/opt/anaconda3/bin:/usr/local/bin:/opt:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
...and remember that that would mean that if you install the same-named utility with both Anaconda and Homebrew then the Anaconda one would be used.

I see. Thank you. I will try that new PATH then.

Yes I usually use capitalized text for things I will just double click and paste something else. The same for my favorite action for ffmpeg:

It's just a "system" I have for a lot of stuff, in and out of KM.

Makes perfect sense, yes.

True. Can I just rename the folder and then rename the action? Or is there anything that I need to change? When I was installing it today, I know I had to do a lot of things in Terminal to update the library files or something like that... Do you think there's something "internally" that I would have to change via Terminal if I rename the folder to just ImageMagick?

alltiagocom · August 27, 2024, 11:42am

I created 2 actions to test this

With ENV_PATH set to
/usr/local/bin:/opt:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin

It displays the ffmpeg information in a window when I run the green action.

When I change the ENV_PATH to
/opt/anaconda3/bin:/usr/local/bin:/opt:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin

and I run the red action, I get this:
2024-08-27 12:41:40 Action 16033924 failed: Execute a Shell Script failed with script error: text-script: line 1: ffmpeg: command not found

That's why it's so confusing to me, because everything you said makes perfect sense, but then it doesn't work.

Nige_S · August 27, 2024, 12:18pm

It looks like you're doing everything right, so the problem isn't with you!

Posting the dialogs from this may reveal something:

ffmpeg Troubleshooting.kmmacros (3.1 KB)

I've always avoided Anaconda myself, since I spend a lot of time at work fixing problems it's caused -- though that's more likely down to our scientists doing it wrong than Anaconda itself If you can remind me which version of macOS you're on I see if I can find a test machine.

alltiagocom · August 27, 2024, 8:46pm

First action:
/opt/anaconda3/bin:/usr/local/bin:/opt:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin

Second action:
Runs, but no window pops up

Don't know if this is the same thing, but if I run "which ffmpeg" in Terminal, I get this:
(base) dannywyatt@Dannys-MacBook-Pro ~ % which ffmpeg
/opt/anaconda3/bin/ffmpeg

alltiagocom · August 27, 2024, 9:00pm

macOS 10.15.7 (Catalina)

Is there a way to stop using ffmpeg with anaconda without causing any issues?
Today I installed MacPorts when I was trying to make ImageMagick work so maybe I can use that to reinstall ffmpeg (and any other packages that maybe anaconda installed?)

I just don't want to make changes that will then mess with any macros I have or any configurations that make ffmpeg work at the moment. Since I don't know anything about these things, I don't know how this works and if making changes would break anything?

I asked ChatGPT if I could install ffmpeg via MacPorts and it said yes, I just need to be careful about the PATH and all that. So, without knowing much about this, here's what I would like to know:
What if I install ffmpeg and any packages I remember installing such as ffmpeg-normalize via MacPorts, then "disable" the PATH for anaconda (in the zshrc and bashprofile files, maybe?), but leave it installed just in case something breaks in the future, and just update the path for ffmpeg and ffmpeg-normalize to use mac ports? Hope it makes sense...?

That way I can start using something that apparently could be more reliable, without uninstalling anaconda, which will be there in case I notice something not working that relies on anaconda. Does it make sense?

alltiagocom · August 27, 2024, 10:11pm

Ok, I installed ffmpeg via MacPorts, did everything that ChatGPT told me to do, updated the .zshrc file to

#MacPorts path
export PATH="/opt/local/bin:$PATH"

homebrew path
export PATH="/usr/local/Homebrew/bin:$PATH"

homebrew shell environment
eval "$(/usr/local/Homebrew/bin/brew shellenv)"

imagemagick path
export MAGICK_HOME="/usr/local/ImageMagick-7.0.10"
export PATH="$MAGICK_HOME/bin:$PATH"
export DYLD_LIBRARY_PATH="$MAGICK_HOME/lib/"

The ENV_PATH is set to

/opt/local/bin:/usr/local/bin:/opt:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin

When I run which ffmpeg in Terminal I get this:
/opt/local/bin/ffmpeg

When I run a simple command in Execute Shell:

ffmpeg -i "/Users/dannywyatt/My Files/Inbox Global/Untitled.wav" "/Users/dannywyatt/My Files/Inbox Global/Untitled.mp3"

I get:
Action 16240278 failed: Execute a Shell Script failed with script error: text-script: line 1: ffmpeg: command not found

So I got back to adding the path to make it work (which in that case, it doesn't make any difference if it's MacPorts or Anaconda)

/opt/local/bin/ffmpeg -i "/Users/dannywyatt/My Files/Inbox Global/Untitled.wav" "/Users/dannywyatt/My Files/Inbox Global/Untitled.mp3"

I just don't understand what's the issue with Keyboard Maestro, if Terminal works as expected...

ComplexPoint · August 28, 2024, 8:48am

You've tried comparing the outputs of:

which ffmpeg

from, respectively:

the environment in your copy of Terminal.app, and
the particular Execute Shell Script environment which you are building up in Keyboard Maestro.app ?

(Old) Issue with Shell, still can't figure out why this isn't working

.zshrc

.bash_profile

Options