Ubuntu – How to turn off Nvidia GPU on a headless server

gpuheadlessnvidiapower-managementserver

I am running a headless server with an Nvidia GPU.
Even when the GPU is not doing any work, it is consuming about 25 Watts of power:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 430.26       Driver Version: 430.26       CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 950     Off  | 00000000:01:00.0 Off |                  N/A |
|  0%   61C    P0    26W / 110W |      0MiB /  2001MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Is there a way to completely turn off power delivery to the GPU when it is not in use?

I tried
sudo prime-select intel
Which does cause nvidia-smi to stop working, but a power meter connect to the wall shows exactly the same power consumption with either intel or nvidia selected.

Completely removing the GPU reduces the power consumption by about 30 Watts as expected.

The main purpose is to save power and costs during idle operations, with an option to spin up the GPU when it is needed (remotely via ssh).

Best Answer

It can be observed that the GPU is using Performance mode P0, where the power consumption of the card is very high. In Idle, it should ideally throttle back to P8 mode, where it will consume less power.

I follow the following steps to change the behaviour. Assuming that you are using Linux:

You would need sudo privileges to the machine. First, enable the persistence mode:

sudo nvidia-smi -pm 1

Then, reset the clocks for the GPU:

sudo nvidia-smi -rgc

Note: To know more about these commands, please type

nvidia-smi --help
Related Question