TensorDock
  • Who we are
    • Welcome to TensorDock
    • Our Ethos and Commitment
  • Virtual Machines
    • How to SSH into your instance
    • How to RDP into your instance
    • Installing NVIDIA Drivers on Windows 10
    • Linux & NVIDIA Drivers
    • Cloud-init
    • File transferring with SCP and Rclone
    • Running Jupyter Notebook
    • SSH server hardening on Ubuntu
    • Running Stable Diffusion in Docker
    • Installing and running Stable Diffusion UI
    • Running Disco Diffusion on a Linux instance
    • Running Disco Diffusion on a Windows instance
    • Running SimpleTuner/Flux on a Linux instance
    • Using Parsec, Moonlight and Sunshine
  • Hosting
    • Installation Guide
  • Whitelabel
    • Overview
    • Setting Up a Storefront
    • Customization Overview
    • Customize Whitelabel Storefront
  • Legal Information
    • Company Information
    • Terms of Service (TOS)
    • Privacy Policy
    • Acceptable Use Policy (AUP)
    • Taxes, VAT, GST
    • Downtime Compensation
    • Supplier Hosting Agreement
  • Quick Links
    • Discord Server
    • TensorDock.com
    • Dashboard
    • API Documentation
Powered by GitBook
On this page
  • Important Note - Holding & Unholding NVIDIA driver versions
  • Important Note - NVIDIA H100 SXM5
  • Installing a new driver
  • 1. Search for your NVIDIA driver
  • 2. Visit the downloads page
  • 3. Copy the driver download link
  • 4. SSH onto your TensorDock instance
  • 5. Download the driver onto your VM
  • 6. Enable execution permissions for the driver installer you just downloaded
  • 7. Run the driver installer
  • 8. Reboot!
  • 9. Confirm everything is working
  • Issues

Was this helpful?

  1. Virtual Machines

Linux & NVIDIA Drivers

PreviousInstalling NVIDIA Drivers on Windows 10NextCloud-init

Last updated 4 months ago

Was this helpful?

This tutorial assumes that you have already deployed a GPU server on the TensorDock platform:

Important Note - Holding & Unholding NVIDIA driver versions

NVIDIA drivers automatically update. Once the drivers update, they require a reboot for the GPUs to become usable again. By default, our templates lock your driver image to the version they were built with so that the GPUs never become unusable.

To unlock the driver version, run the following command:

sudo apt-mark unhold nvidia* libnvidia*

Once you upgrade to a new driver version, you can lock the new driver version to prevent the driver from updating automatically in the future. Run the following command as the root user to do this.

dpkg-query -W --showformat='${Package} ${Status}\n' | grep -v deinstall | awk '{ print $1 }' | grep -E 'nvidia.*-[0-9]+$' | xargs -r -L 1 apt-mark hold

Important Note - NVIDIA H100 SXM5

Our NVIDIA H100 SXM5 servers require the installation of the nvidia-fabricmanager-535 driver for the GPU driver to properly use the NVSwitch fabric installed. NVLink is only enabled for 8x H100 VMs. If you do not install this package, CUDA will NOT work properly.

Our TensorML operating system packages include this package, but our base templates do not.

First, we'll need to unhold the default drivers included with our operating system templates:

sudo apt-mark unhold nvidia* libnvidia*

Then, we'll need to install the NVSwitch FabricManager package:

sudo apt update
sudo apt install nvidia-fabricmanager-535

Finally, we'll upgrade all of our packages before rebooting, which will bring the GPU driver up to date with the FabricManager package.

sudo apt upgrade -y 
sudo reboot

As pictured, nvidia-smi -q should show Fabric State = Completed after the reboot. This indicates the GPUs are ready for usage!

Installing a new driver

1. Search for your NVIDIA driver

First, search for your GPU through the link below and copy the link to the NVIDIA driver.

For instance, for a GeForce 4090:

  • Product Type: GeForce

  • Product Series: GeForce RTX 40 Series

  • Product: NVIDIA GeForce RTX 4090

  • Operating System: Linux 64-bit

  • Download Type: Production Branch

  • Language: English (US)

2. Visit the downloads page

Once you get redirected to the driver, click on the "Download" button. Don't worry; it won't actually initiate a download. It will simply redirect you to a page where you'll confirm NVIDIA's EULA.

3. Copy the driver download link

Now, you can copy the link to the actual driver.

4. SSH onto your TensorDock instance

Use the port forwarded into port 22 as your SSH port. You should see something like the following:

5. Download the driver onto your VM

Use wget and then append the driver's URL. This will save the driver in whatever directory you're in.

6. Enable execution permissions for the driver installer you just downloaded

Run chmod +x and then append the file name

7. Run the driver installer

Run sudo ./[DRIVER_FILENAME]

8. Reboot!

Complete the questionarie, and then run sudo reboot to reboot your virtual machine!

9. Confirm everything is working

Now, nvidia-smi should work!

Issues

wget https://us.download.nvidia.com/XFree86/Linux-x86_64/525.60.11/NVIDIA-Linux-x86_64-525.60.11.run
chmod +x NVIDIA-Linux-x86_64-525.60.11.run
sudo ./NVIDIA-Linux-x86_64-525.60.11.run
sudo reboot

If you're still facing issues, come email us at . For reference, these were the commands we ran while making this tutorial:

support@tensordock.com
New Server Configuration - TensorDock Marketplace
Download the latest official NVIDIA driversNVIDIA
Click on this link to search for the NVIDIA driver for your graphics card
Logo
Logo
When NVIDIA drivers automatically update, the GPUs becomes unusable. Thus, you should always lock a working driver version
Whoops, nvidia-smi doesn't work! Downloading new drivers will fix that...
Page cover image