MacOS High Sierra python3 / pip awscli install path confusion

high sierrahomebrewmacospython

Was playing around with brew install asciinema and it broke my python install on macOS High Sierra (10.13.3 (17D47)).

But now that I'm re-installing python for the awscli via pip, it's not acting like I would expect.

The Confusion:

The awscli install instructions state "pip installs executables to the same folder that contains the Python executable. Add this folder to your PATH variable." When I do ls -al $(which python), I get

lrwxr-xr-x  1 notbrain  admin  35 Mar  6 10:35 /usr/local/bin/python -> ../Cellar/python/3.6.4_3/bin/python

Which is what I would expect after a brew install. So I should add /usr/local/Cellar/python/3.6.4_3/bin to my path, right?

But aws does not get installed there, it's in

~/Library/Python/3.6/bin

Adding to the confusion – the shebang in the aws entry script is set to:

17:11:40 notbrain@brian ~/Library/Python/3.6/bin
➜ cat aws
#!/usr/local/opt/python/bin/python3.6
# Copyright 2012 Amazon.com, Inc. or its affiliates. All Rights Reserved.

Which is fine, since it's a symlink to the brew-installed python 3.6. But why so many inconsistencies? Where is pip picking up the ~/Library/Python path? Seems like that's the odd macOS specific thing.

Is there something on my system that is set incorrectly to cause this discrepancy? Is this a pip install config thing I could change to work with homebrew installed python 3?

Install Steps

I removed all python installed by brew and get the system installed 2.7:

10:20:20 notbrain@brian ~
➜ python --version
Python 2.7.10

10:20:34 notbrain@brian ~
➜ which python
/usr/bin/python

10:20:42 notbrain@brian ~
➜ ll $(which python)
.rwxr-xr-x 66k root  1 Dec  2017 /usr/bin/python

10:21:13 notbrain@brian ~
➜ whereis python
/usr/bin/python

Then a reinstall of python with homebrew:

10:26:17 notbrain@brian ~
➜ brew install python
==> Downloading https://homebrew.bintray.com/bottles/python-3.6.4_3.high_sierra.bottle.tar.gz
Already downloaded: /Users/notbrain/Library/Caches/Homebrew/python-3.6.4_3.high_sierra.bottle.tar.gz
==> Pouring python-3.6.4_3.high_sierra.bottle.tar.gz
==> /usr/local/Cellar/python/3.6.4_3/bin/python3 -s setup.py --no-user-cfg install --force --verbose --install-scripts=/usr/local/Cellar/python/3.6.4_3/bin --install-lib=/usr/local/lib/
==> /usr/local/Cellar/python/3.6.4_3/bin/python3 -s setup.py --no-user-cfg install --force --verbose --install-scripts=/usr/local/Cellar/python/3.6.4_3/bin --install-lib=/usr/local/lib/
==> /usr/local/Cellar/python/3.6.4_3/bin/python3 -s setup.py --no-user-cfg install --force --verbose --install-scripts=/usr/local/Cellar/python/3.6.4_3/bin --install-lib=/usr/local/lib/
==> Caveats
Pip, setuptools, and wheel have been installed. To update them
  pip3 install --upgrade pip setuptools wheel

You can install Python packages with
  pip3 install <package>

They will install into the site-package directory
  /usr/local/lib/python3.6/site-packages

See: https://docs.brew.sh/Homebrew-and-Python

Unversioned symlinks python, python-config, pip etc. pointing to python3,
python3-config, pip3 etc., respectively, have been installed.

If you need Homebrew's Python 2, `brew install python@2`.
==> Summary
?  /usr/local/Cellar/python/3.6.4_3: 3,615 files, 56.

10:26:44 notbrain@brian ~
➜ which python
/usr/local/bin/python

And finally, a pip install of awscli using https://docs.aws.amazon.com/cli/latest/userguide/cli-install-macos.html#awscli-install-osx-pip

10:35:49 notbrain@brian ~
➜ pip install awscli --upgrade --user
Collecting awscli
  Using cached awscli-1.14.50-py2.py3-none-any.whl
Collecting docutils>=0.10 (from awscli)
  Using cached docutils-0.14-py3-none-any.whl
Collecting colorama<=0.3.7,>=0.2.5 (from awscli)
  Using cached colorama-0.3.7-py2.py3-none-any.whl
Collecting s3transfer<0.2.0,>=0.1.12 (from awscli)
  Using cached s3transfer-0.1.13-py2.py3-none-any.whl
Collecting botocore==1.9.3 (from awscli)
  Using cached botocore-1.9.3-py2.py3-none-any.whl
Collecting rsa<=3.5.0,>=3.1.2 (from awscli)
  Using cached rsa-3.4.2-py2.py3-none-any.whl
Collecting PyYAML<=3.12,>=3.10 (from awscli)
Collecting python-dateutil<3.0.0,>=2.1 (from botocore==1.9.3->awscli)
  Using cached python_dateutil-2.6.1-py2.py3-none-any.whl
Collecting jmespath<1.0.0,>=0.7.1 (from botocore==1.9.3->awscli)
  Using cached jmespath-0.9.3-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3 (from rsa<=3.5.0,>=3.1.2->awscli)
  Using cached pyasn1-0.4.2-py2.py3-none-any.whl
Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore==1.9.3->awscli)
  Using cached six-1.11.0-py2.py3-none-any.whl
Installing collected packages: docutils, colorama, six, python-dateutil, jmespath, botocore, s3transfer, pyasn1, rsa, PyYAML, awscli
Successfully installed PyYAML-3.12 awscli-1.14.50 botocore-1.9.3 colorama-0.3.7 docutils-0.14 jmespath-0.9.3 pyasn1-0.4.2 python-dateutil-2.6.1 rsa-3.4.2 s3transfer-0.1.13 six-1.11.0

Best Answer

Did you ever resolve? I'm also kind of stuck trying to get awscli running. I think it's added to my path because I see this: $ echo ${PATH} usr/Library/Python/3.7/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

and I see the aws executable files in that first directory in the path, but no aws commands are found when I try to use it

EDIT: this is the path addition that finally worked: export PATH=~/Library/Python/3.7/bin:$PATH

I found the solution here: https://dev.to/stevoperisic/installing-python3-pip-and-aws-cli-on-macos-sierra-3pea

Related Question