Ubuntu – Installing software with python pip

python

I know that building software from source on my system using root privileges usually causes various problems and that the preferred way to install anything is to use a packet management system (apt-get first, dpkg second).

However, I need some python programs which can be only installed with pip. I have tried to use pip install --install-option="--prefix=/apps/", but it seems that also in that case the pip wants to write to /lib/python2.7/ which is a system folder.

Is there a way to install software with pip without the possibility of damaging or polluting my system with unmanaged files?

Best Answer

Python virtual environment creator (python-virtualenv) allows you to create a sandboxed and isolated environment where Python packages can be installed without interfering with other packages on the same machine. With several virtualenvs, many different pieces of Python software with different and even mutually exclusive dependencies can coexist together. You can install python-virtualenv from the default Ubuntu repositories in all currently supported versions of Ubuntu.

Set up a virtual environment for Python anywhere in your home directory, activate your Python virtual environment from the terminal, and then install whatever packages you need to be installed by pip locally in your virtual environment as a normal user using pip install.

Install Python virtual environment creator (virtualenv):
```
sudo apt install python-virtualenv virtualenv  
```
Make a new directory (I'll call it PythonVirtualEnv in this example) for the Python virtual environment and setup the Python virtual environment with Python and pip in it.

    cd ~  
    mkdir PythonVirtualEnv
    virtualenv PythonVirtualEnv

Install some packages.

    cd ~/PythonVirtualEnv  
    source bin/activate
    python -m pip install <insert-name-of-package-here>

Deactivate the Python virtual environment before leaving it.
```
deactivate  
```

Creating an environment with a custom Python interpreter

sudo apt install python3-virtualenv 
cd ~  
mkdir Python3VirtualEnv
virtualenv --python=/usr/bin/python3 Python3VirtualEnv # /usr/bin/python3 is the default location of the python3 executable
cd ~/Python3VirtualEnv  
source bin/activate
python3 -m pip install <insert-name-of-package-here>

Related Solutions

Ubuntu – trying to install thesql-connector for python 3

Ah! The problem you seem to be facing is you can't create virtualenv with python3.

And to install dependencies for python3 environment you run pip3 install package-name

Follow these steps for creating virtualenv in python3 environment:

virtualenv -p /usr/bin/python3 yourenv
source yourenv/bin/activate
pip3 install mysql-connector

Ubuntu – apt-get install python-xxx vs pip install

For the archives, after much trial and error this is what I've found (in retrospect, some bits make sense, others quite confusing).

If you sudo apt-get install python-xyz, the package will appear in pip list. Of course you can import it in python, but it doesn't appear in pip.get_installed_distributions(). (Obviously the opposite is not true. i.e. if you do pip install xyz it will not appear in your apt/synaptic)
After sudo apt-get install python-xyz if you do pip install xyz what happens depends on the version of pip you have.

OLD v1.5.6 (the current version shipping on Ubuntu PPA):

This version of pip (1.5.6) will just install a new copy of xyz in a different location. You end up with multiple copies and this causes a huge mess. e.g:
- numpy from apt-get is 1.8.2 at /usr/local/lib/python2.7/dist-packages
- numpy from pip is 1.10.4 at ~/.local/lib/python2.7/site-packages
If I do pip install numpy yet again, it downloads and installs it again. So you could end up with many different versions which you can't really access. I can do pip install numpy 5x times, and then I can do pip uninstall numpy 5x times! Obviously you can pay attention to not do that, but sometimes other software's install scripts are a bit careless and can mess things up. Unbelievable that Ubuntu officially ships this version of pip.

NEW v8.0.2 (the current version on pip itself):

newer versions of pip (e.g. 8.0.2 which is on pip) will refuse to install the same package saying requirement is already met. So you cannot install a new version. This is good behaviour (more on getting this version of pip later).
In this case you can only upgrade, i.e. install with -U flag.
However when you try to pip install -U xyz on a package installed with sudo apt-get, you'll get permissions error because the apt-get was installed in /usr/ and you need root access to write there.
So AFAIK you have no choice but to sudo pip install -U xyz to be able to update it. In this case pip installs the latest package into the same place as sudo apt-get wrote it. e.g. in my case /usr/local/lib/python2.7/dist-packages. This is good.
It's also worth pointing out that the packages on apt are often quite older than those on pip (e.g. numpy v1.8.2 vs 1.10.4, scipy 0.14.1 vs 0.17.0, ipython 2.3 vs 4.0.3, spyder 2.3.5 vs 2.3.8)

So my current thoughts are to get the big things with sudo apt-get, e.g. sudo apt-get install python-numpy python-scipy python-matplotlib ipython ipython-notebook python-pandas python-sympy python-nose spyder

and then update them (or at least some of them) with sudo pip install -U.

NB It seems quite important to get the new pip from pip (very meta)

sudo apt-get install python-pip
sudo pip install -U pip

P.S. I am aware of virtualenv but I have no need for it right now. I need only one development environment.

And here is a little script to dump a list of packages, version and path (but only works on pip installed modules, not those from apt-get)

import pip
pp = pip.get_installed_distributions()
for p in sorted([p.location+"\t"+p.project_name+" ("+p.version+")" for p in pp]):
    print p

Best Answer

Creating an environment with a custom Python interpreter

Related Solutions

Ubuntu – trying to install thesql-connector for python 3

Ubuntu – apt-get install python-xxx vs pip install

Related Question