Setting Up NVIDIA CUDA on Ubuntu 26.04 LTS for AI/ML Workloads

We’d been running our training rig on 24.04 for a while, mostly because “if it works, don’t touch it.”

But once we moved it over to 26.04, the GPU setup felt noticeably different from the usual routine. Fewer manual steps, less googling error messages halfway through.

Here’s the actual Ubuntu 26.04 CUDA setup we ran, start to finish, plus the errors we hit along the way.

Why 26.04 Actually Matters Here

This isn’t just a version bump for the sake of it.

Canonical’s own release announcement for 26.04 LTS specifically calls out native support for NVIDIA CUDA and AMD ROCm, built directly into the platform rather than bolted on afterward. That’s a real shift from how GPU setup has worked on Ubuntu for years.

Previously, getting CUDA running meant driver installation, then a separate trip to NVIDIA’s developer site, then adding their repository by hand, then hoping the versions lined up correctly. It worked, but it was never exactly smooth.

On 26.04, the driver detection and CUDA toolkit installation both move through the same familiar apt workflow, with far less manual repository wrangling than before. That single change is why this Ubuntu 26.04 CUDA setup felt lighter than every previous attempt we’d made on older releases.

There’s also the ROCm side worth mentioning briefly, even if we’re focused on NVIDIA here. Canonical is maintaining ROCm packages directly for AMD hardware now too, which means both major GPU ecosystems get first-party treatment on this release, not just NVIDIA. If you’re choosing hardware for a new AI/ML box, that’s a meaningful shift in how much setup friction you’re signing up for either way.

Checking What You’re Working With

Every Ubuntu 26.04 CUDA setup starts the same way, so before installing anything, we confirmed the GPU was actually detected.

lspci | grep -i nvidia

01:00.0 VGA compatible controller: NVIDIA Corporation AD104 [GeForce RTX 4070] (rev a1)

Good, the card was visible at the hardware level. Next, we checked what Ubuntu’s driver manager recommended.

sudo ubuntu-drivers devices

== /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0 ==
model     : GeForce RTX 4070
driver    : nvidia-driver-580 - distro non-free recommended

That’s the driver Ubuntu itself suggests, based on the actual hardware, not a generic default.

Installing the Driver

We let Ubuntu handle driver selection automatically, rather than picking a specific version by hand.

sudo ubuntu-drivers autoinstall

This pulled in the recommended proprietary driver along with the correct kernel modules for it. Once it finished, we rebooted.

sudo reboot

After the reboot, a quick check confirmed the driver had loaded correctly.

nvidia-smi

+-----------------------------------------------------------------------+
| NVIDIA-SMI 580.xx    Driver Version: 580.xx    CUDA Version: 13.x     |
|-------------------------------+----------------------+----------------|
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr|
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M.|
|===============================+======================+================|
|   0  GeForce RTX 4070    Off  | 00000000:01:00.0  On  |          N/A   |
+-------------------------------+----------------------+----------------+

Driver installed, GPU recognized, CUDA version visible right there in the header. That’s the first real checkpoint.

Installing the CUDA Toolkit for This Ubuntu 26.04 CUDA Setup

The driver alone gets you nvidia-smi, but not nvcc, the compiler tools most ML frameworks and CUDA-based projects actually depend on during build steps.

We added NVIDIA’s repository directly, following the official CUDA installation guide:

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2604/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update

Then installed the toolkit itself:

sudo apt install cuda-toolkit

This pulled in a fairly large set of packages, compilers, libraries, profiling tools, so give it a few minutes depending on your connection.

Once it finished, we checked the compiler directly:

nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver
Cuda compilation tools, release 13.3, V13.3.107

That confirms the toolkit itself, separate from just the driver, is correctly in place.

Verifying the Ubuntu 26.04 CUDA Setup With PyTorch

Driver and toolkit installed is one thing. Actually being usable from a real ML framework is the part that matters, and it’s the same check we run when standing up a private model API.

We installed PyTorch with CUDA support in a fresh virtual environment:

python3 -m venv cuda-test
source cuda-test/bin/activate
pip install torch --index-url https://download.pytorch.org/whl/cu130

Then ran a quick check:

import torch

print(torch.cuda.is_available())
print(torch.cuda.get_device_name(0))

True
NVIDIA GeForce RTX 4070

True, and the correct card name printed back. That’s the real confirmation that this Ubuntu 26.04 CUDA setup is actually usable for training or inference, not just installed and sitting idle.

If you’re on TensorFlow instead, the equivalent check is just as short:

import tensorflow as tf

print(tf.config.list_physical_devices('GPU'))

[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

Either output confirms the same thing, your framework can actually see and use the GPU, not just that the driver technically loaded.

Common Errors We Hit (and the Fixes)

Most Ubuntu 26.04 CUDA setup problems fall into a handful of buckets. Here’s a short list of what actually went wrong along the way, since a clean run-through rarely tells the whole story.

nvidia-smi returns “command not found” after reboot The driver install likely didn’t complete, or you rebooted before it finished. Re-run sudo ubuntu-drivers autoinstall and check dpkg -l | grep nvidia-driver for the package before rebooting again.

Secure Boot blocks the driver from loading If Secure Boot is enabled, the proprietary NVIDIA kernel module may refuse to load unless it’s signed. Either disable Secure Boot in your BIOS, or enroll a Machine Owner Key (MOK) when prompted during the driver install.

torch.cuda.is_available() returns False despite nvidia-smi working This almost always means a CUDA version mismatch between the PyTorch build and your installed toolkit. Double-check you installed the PyTorch wheel matching your actual CUDA version, not just the latest one by default.

apt install cuda-toolkit fails with unmet dependencies Usually caused by a partial or conflicting older CUDA install still hanging around. Purge old packages first with sudo apt remove --purge 'cuda-*' 'nvidia-cuda-toolkit', then retry.

GPU detected by lspci but missing from ubuntu-drivers devices Run sudo apt update first, the driver database itself may just be stale on a freshly installed system.

Was This Ubuntu 26.04 CUDA Setup Worth It?

For us, genuinely yes.

The whole Ubuntu 26.04 CUDA setup, from a fresh install to a working, verified GPU environment, took under twenty minutes with driver and toolkit both included. That’s a real difference from the multi-step, cross-reference-the-website process this used to be.

If you’re setting up a new machine for AI or ML work and haven’t tried the newer Ubuntu 26.04 CUDA setup path yet, it’s worth doing on 26.04 specifically rather than falling back to the old manual routine out of habit. The native integration Canonical built in here actually holds up in practice, not just in the release notes.

For more Ubuntu walkthroughs, browse our Ubuntu guides.