The suggested way to handle pip(easy_install) with homebrew

easy-installhomebrewpippython

I know there are brew-gem and brew-pip but it is still really easy to get confused. Let's say my Mac OS X is 10.7.2.

  • There are at least, as far as I know, 3 locations for Python modules (assume 2.7):
    1. /System/Library/Frameworks/Python.framework/Versions/2.7/
    2. /Library/Python/2.7/site-packages
    3. /usr/local/lib/python2.7/site-packages/ (controlled within homebrew)
  • For some Python modules, pip install them into 2, the so-called local/customized Python module location, and everything looks and works great. Ex, readline by easy_install (ipython suggested me to install readline by easy_install instead of pip)
  • For some, it would try to install some miscellaneous files (ex, man, doc, …) into system-wide location, which requires sudo! Ex, ipython insisted on installing man and doc into /System/Library/Frameworks/Python.framework/Versions/2.7/share/, which violates permission issue and all I can do is to use sudo.
  • For some Python modules installed by brew, they are symbolic linked to /usr/local/lib/python2.7/site-packages/. Everything seems great except that you have to remember to add this location into PYTHONPATH.

I am wondering any suggested and uniform way to handle those mass, or any explanation to make those stuff crystal clear.

Best Answer

Maybe it is time to consider using virtualenv. Virtualenv creates self-contained python environments using the python version you specify. After activating the new virtual environment, everything you install using pip goes under that environment. This helps avoid situations like the one you described.

E.g. create and activate a new python environment using the default python:

# create environment
$ virtualenv --distribute myproject
New python executable in myproject/bin/python
Installing distribute...done.
Installing pip...done.

# activate environment
$ . ./myproject/bin/activate

# check default python
(myproject)$ which python
/Users/me/myproject/bin/python

It is suggested to use the --distribute options to indicate that distribute should be used for installing packages in the new environment instead of (the older) setuptools. After activation your command prompt changes to indicate which python environment is active.

Now install some package. The files will go into myproject directory:

# install django
(myproject)$ pip install django
...

# search for django dir
(myproject)$ find myproject -iname django
myproject/lib/python2.7/site-packages/django

Finally, deactivate:

# deactivate and check for default python
(myproject)$ deactivate
$ which python
/usr/bin/python

To create an environment using a non-default version of python:

$ virtualenv --distribute -p /path/to/custom/python mynewproject

By default virtualenv will copy to the new environment any packages installed for the python version you use to bootstrap it. To prevent this and create an empty environment use the --no-site-packages option. This is especially useful to create environments which can be exactly replicated e.g. from development to production.

Update: As of version 1.7 --no-site-packages has become the default behaviour of virtualenv.

If you want more details, there are plenty of tutorials and blog posts online. E.g.:

  1. Notes on using pip and virtualenv with Django. (most of the post is not django-specific)
  2. Working with virtualenv.

Give it a try and I'm sure you'll stick with it.

Note: Make sure that your executable scripts do not have the python interpreter hardcoded. I.e. their first line should be #!/usr/bin/env python and not something like #!/usr/bin/python.

Related Question