Translations of this page:
In order to see what are the GPU's available
nvidia-smi
Sometimes nvidia-smi show that a GPU is available, while it is inaccessible. It is most of the time due to GPU process not killed properly. You can list the processes related to the GPU's by typing
lsof /dev/nvidia*
Then you can kill the process bonded to the GPU.
kill ProcessPIDNumber
PS: Sometimes you have to repeat this command to ensure that the process has been completely terminated.
If the server have 4 GPUs [0,1,2,3]
CUDA_VISIBLE_DEVICES=GPU_number python myprogram.py
ex
CUDA_VISIBLE_DEVICES=1 python myprogram.py
If no CUDA_VISIBLE_DEVICES is specified GPU 0 will be used by default.