How to install CUDA and find the right PyTorch to go with it.

Getting a GPU working with PyTorch is a rite of passage, and it's harder than it should be for one reason: the word "CUDA" refers to three different things, and most guides don't say which one they mean. Untangle that, and the installation becomes a five-minute job. This guide does the untangling first, then the installing.

The three things called "CUDA"

• The NVIDIA driver — the low-level software that lets your operating system talk to the GPU at all. Installed system-wide, versioned like 580.xx.
• The CUDA runtime libraries — the libraries a program like PyTorch uses to launch work on the GPU (cudart, cuBLAS, cuDNN, and friends). Versioned like 12.6.
• The CUDA Toolkit — the full development kit, including the nvcc compiler, headers, and profilers. Also versioned like 12.6, but a much bigger install.

Now the fact that saves you an afternoon:

PyTorch's pip wheels bundle their own CUDA runtime libraries. To run PyTorch on a GPU you need exactly one thing installed system-wide: a sufficiently recent NVIDIA driver. You do not need to install the CUDA Toolkit from NVIDIA's website unless you plan to compile CUDA code yourself.

Most broken setups come from people fighting with toolkit installers they never needed, or from mismatched expectations about what nvidia-smi reports — which brings us to step one.

Step 1: check your driver with nvidia-smi

Open a terminal and run:

nvidia-smi

The header of the output looks like this:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 580.65      Driver Version: 580.65      CUDA Version: 13.0       |
+-----------------------------------------------------------------------------+

Here is the single most misunderstood line in deep learning setup: that "CUDA Version" is not the CUDA version you have installed. It is the maximum CUDA runtime version your current driver supports. In the example above, the driver can run applications built against CUDA 13.0 or anything older — drivers are backward compatible with earlier CUDA runtimes.

So the rule for choosing a PyTorch build is simply:

Pick any PyTorch CUDA variant whose version is ≤ the "CUDA Version" shown by nvidia-smi.

If nvidia-smi isn't found, or the supported CUDA version is old, install or update the driver first:

• Windows — install the current GeForce/Studio driver from nvidia.com/drivers (or via GeForce Experience / NVIDIA App). Reboot.
• Linux — use your distribution's packaged driver (e.g. on Ubuntu: sudo ubuntu-drivers install, or sudo apt install nvidia-driver-580 for a specific series). Reboot.
• WSL2 — install the Windows driver only. Do not install a Linux display driver inside WSL; the Windows driver exposes the GPU to WSL automatically, and nvidia-smi will work inside your Linux shell.

Step 2: install the matching PyTorch build

Go to the official selector at pytorch.org/get-started/locally, pick your OS, package manager, and a CUDA version your driver supports, and it generates the exact command. The commands it produces look like this:

# PyTorch built against CUDA 12.6 (works with any driver supporting CUDA ≥ 12.6)
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu126

# Older driver? Pick a lower CUDA build instead, e.g. CUDA 11.8
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118

# No NVIDIA GPU at all: the CPU-only build
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu

The cu126-style suffix is the CUDA runtime version the wheel was built against. The exact set on offer changes with each PyTorch release — older CUDA builds are retired and new ones added — which is why the selector page, not a blog post, should always be your source for the literal command. The logic for choosing, however, doesn't change: highest CUDA variant that is ≤ your driver's supported version.

Windows pitfall: a plain pip install torch (without the --index-url) installs a CPU-only build on Windows. If torch.cuda.is_available() returns False on a Windows machine with a healthy GPU, this is the most likely reason. On Linux, the default PyPI wheel does ship with CUDA support.

Using conda? Recent PyTorch releases have moved to pip as the primary distribution channel, so the cleanest pattern is a conda environment with PyTorch pip-installed inside it using the same commands as above.

Step 3: verify

The critical test is as follows:

import torch

print(torch.cuda.is_available())      # True, if all is well

You can also run:

print(torch.__version__)              # e.g. 2.7.1+cu126
print(torch.version.cuda)             # CUDA runtime bundled with this build, e.g. 12.6
print(torch.cuda.get_device_name(0))  # e.g. NVIDIA GeForce RTX 4070

When you do need the full CUDA Toolkit

Everything above covers running PyTorch. You need the actual CUDA Toolkit (with nvcc) only when something must be compiled against CUDA on your machine — typically custom CUDA extensions or packages built from source, such as flash-attn or exotic kernels from research code. In that case:

• Install a toolkit whose major.minor version matches the CUDA version of your PyTorch build (check torch.version.cuda) — compiling an extension against a different CUDA version than PyTorch's is a classic source of import errors.
• Get it from NVIDIA's CUDA Toolkit archive (multiple versions can coexist under /usr/local/cuda-XX.X on Linux), or install it into a conda environment via the nvidia channel's cuda-toolkit package to keep it project-local.
• Verify with nvcc --version — this, unlike nvidia-smi, does report an installed toolkit.

Troubleshooting

The short version

1. Install/update the NVIDIA driver; check nvidia-smi.
2. The "CUDA Version" it reports is a ceiling, not an installed version.
3. From pytorch.org, install a PyTorch build at or below that ceiling.
4. Verify with torch.cuda.is_available().
5. Skip the CUDA Toolkit unless you're compiling CUDA code yourself.

All posts