How to kill all processes using a given GPU

gpukill

I use the CUDA toolkit to perform some computations on my Nvidia GPUs. How to kill all processes that use a given GPU? (killing at once, i.e. without having to manually type the PIDs behind kill -9.)

E.g. killing all processes using GPU 2:

enter image description here

Best Answer

Following the Unix philosophy, you have a tool that lists processes using a given GPU, and a tool that kills processes. Combine them using shell constructs and text processing tools.

For example, to kill all the processes using GPU 2, you can execute the following command:

kill $(nvidia-smi | awk '$2=="Processes:" {p=1} p && $2 == 2 && $3 > 0 {print $3}')

or

kill $(nvidia-smi -g 2 | awk '$2=="Processes:" {p=1} p && $3 > 0 {print $3}')
Related Question