Unable to install NVIDIA driver on various GCP Ubuntu VM s with Tesla K80 GPU

Question

I have followed the guide on GCP below with Ubuntu 18 and 20 ( have also tried Ubuntu Lite , Debian and Centos 7 ):

https://cloud.google.com/compute/docs/gpus/install-drivers-gpu

Unfortunately, after completing the lengthy install I get this:

me@gpu:~$ nvidia-smi
NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running

Have tried installing via the script and via the direct downloads from Nvidia site for Cuda 10. Ready to pull my hair out if that helps! I don't understand how a company that builds a bazillion gpu's can't make the installation process robust?

Answer 1

If you've installed the driver so many times and nvidia-smi is still failing to communicate. Take a look into prime-select

1. Run prime-select query this way you are going to get all possible options, it should show at least nvdia | intel

2. Select prime-select nvidia

3. Then if you see nvidia is already selected, choose a different one, eg. prime-select intel. Next switch back to nvidia prime select nvidia

4. Reboot and check nvidia-smi.

Plus it could also be a good idea to run sudo apt install nvidia-cuda-toolkit.

When it finishes, reboot the machine and nvidia-smi should work.