What happens in the terminal doesn't actually provide any insight into what happens in an Execute shell action.
You will find, for example, that result of
echo $PATH
in Terminal.app
is not the same as the result of the same line in an Execute Shell Script actions.
The two shell instances have no shared history, and no knowledge of each other.
You have to explicitly specify the path to the pandas module, or explicitly add that path to the the PATH variable at action run-time