Ubuntu – NVIDIA card for CUDA and AMD card for display on Ubuntu 16.04

cudahybrid-graphicsnvidia

I am using Ubuntu 16.04, since it is the OS that I found more convenient for working with convolutional neural networks (CNN). The process of installing Ubuntu was fine until I wanted to setup two graphic cards on the same system:

  • NVIDIA 970 GTX
  • AMD R9 380

I wanted to use my NVIDIA card only for CUDA in order to be able to use all its graphic memory and the AMD for the graphical user interface. To this end all monitors will be plugged into this card.

Problems:

  • When I install the nvidia and cuda 8.0 drivers all seems OK, but when
    I install the AMD driver I can not login to lightdm.
  • If I plug the monitors on the AMD cards it says that lightdm is
    running in low graphic mode and nothing work.
  • Modify the xorg.conf file do not makes any difference since it is
    overwritten by the gpu-manager

I could not find any tutorial explaining how to do so.

Best Answer

Before starting I recommend to install ssh in order to be able to shutdown the system remotely if something goes wrong. To do so use this command:

sudo apt-get install ssh

and to shutdown your system you have to ssh with another computer or mobile device and use this command:

sudo shutdown -r now

Step 1

First of all you have to remove the AMD card physically, keep your NVIDIA card and install CUDA and the NVIDIA driver. You can download cuda from here: https://developer.nvidia.com/cuda-downloads

Make sure that all is working using this command:

nvidia-smi

Step 2

At this point we have to modify grub in order to start in console mode since we are messing with graphic cards.

Using your favorite editor (in my case joe) edit the default grub configuration file:

sudo joe /etc/default/grub

And change these lines:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX=""
#GRUB_TERMINAL=console

to these ones:

GRUB_CMDLINE_LINUX_DEFAULT="text"
GRUB_CMDLINE_LINUX="text"
GRUB_TERMINAL=console

Finally update grub using this command:

sudo update-grub

Step 3

Shutdown your system and remove the NVIDIA card and plugin the AMD card. Turn on your computer and install the latest AMD graphic drivers. In my case I installed the drivers using these commands:

wget https://www2.ati.com/drivers/linux/ubuntu/amdgpu-pro-16.40-348864.tar.xz
tar -Jxvf amdgpu-pro-16.40-348864.tar.xz
cd amdgpu-pro-16.40-348864/
./amdgpu-pro-install
sudo usermod -a -G video $USER

Restart your computer. Now you can start lightdm (the user interface) using this command:

sudo service lightdm start

All should work as expected and now you can setup your displays using the system settings.

Step 4

Shutdown your system and now add your NVIDIA card and do not remove the AMD (at this points you will have two graphic cards in your system). Turn on you computer and log in using your user name but DO NOT START LIGHTDM!!

At this point the gpu-manager has automatically setup both graphic cards for the console mode and only AMD for lightdm (since we started lightdm in the previous step). If we run lightdm at this point gpu-manager will make a mess. So now we have to completely disable the gpu-manager. To this end we have to modify grub again:

sudo joe /etc/default/grub

and replace these lines:

GRUB_CMDLINE_LINUX_DEFAULT="text"
GRUB_CMDLINE_LINUX="text"
GRUB_TERMINAL=console

by these ones:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash nogpumanager"
GRUB_CMDLINE_LINUX=""
#GRUB_TERMINAL=console

Finally update grub using this command:

sudo update-grub

Step 5

Restart your system and if the lightdm do not start correctly (it appears a black screen) then you have to press Ctrl+F1, login in console mode and add this command to the rc.local file:

sudo joe /etc/rc.local

add this line before "exit 0":

service lightdm start

restart your system and all should work fine now ...

In a console you can check that both graphics cads are in use using this command:

lspci -nnk | grep -i vga -A3 | grep 'in use'

Kernel driver in use: amdgpu
Kernel driver in use: nvidia

and nvidia-smi should work fine. showing that you are using 0 megabytes of memory:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 375.26                 Driver Version: 375.26                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 970     Off  | 0000:02:00.0     Off |                  N/A |
| 30%   42C    P0    36W / 163W |      0MiB /  4036MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
Related Question