{
  "schemaVersion": "1.0",
  "entity": "BlogPosting",
  "title": "How to Install NVIDIA Container Toolkit & PyTorch CUDA (2026)",
  "description": "Complete guide to installing NVIDIA GPU drivers, NVIDIA Container Toolkit (nvidia-ctk), and PyTorch CUDA 12 on Ubuntu 24.04/22.04 LTS.",
  "author": "vd",
  "datePublished": "2026-07-24T00:00:00.000Z",
  "dateModified": "2026-07-24T00:00:00.000Z",
  "tags": [
    "HowTo",
    "NVIDIA",
    "CUDA",
    "PyTorch",
    "Docker",
    "AI",
    "Linux"
  ],
  "aeoDirectAnswers": [
    {
      "question": "What are the system requirements for PyTorch CUDA hardware acceleration?",
      "answer": "Running PyTorch with CUDA hardware acceleration requires an Ubuntu or Debian Linux system with an active NVIDIA GPU, proprietary NVIDIA display drivers, and Docker Engine 24.0+ installed. Ensure your workstation or cloud server satisfies these baseline hardware and software prerequisites: **Operating System:** Ubuntu 24.04 LTS, 22.04 LTS, or Debian 12"
    },
    {
      "question": "How do you install NVIDIA GPU drivers on Ubuntu?",
      "answer": "To install NVIDIA GPU drivers on Ubuntu, run ubuntu-drivers devices to inspect recommended driver packages and install nvidia-driver-550 or higher using the apt package manager. Before configuring container runtimes, install the matching NVIDIA proprietary Linux driver."
    },
    {
      "question": "How do you install the NVIDIA Container Toolkit (`nvidia-ctk`)?",
      "answer": "To install the NVIDIA Container Toolkit, add the official NVIDIA GPG key and repository list to APT, install nvidia-container-toolkit, and run nvidia-ctk runtime configure --runtime=docker. The NVIDIA Container Toolkit enables Docker containers to share host GPU resources using --gpus all."
    },
    {
      "question": "How do you install PyTorch with CUDA 12 support?",
      "answer": "To install PyTorch with CUDA 12 support, create a Python virtual environment and run pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124. Install PyTorch compiled against CUDA 12 libraries within an isolated Python virtual environment."
    },
    {
      "question": "How do you restrict PyTorch to specific GPU devices?",
      "answer": "To restrict PyTorch to specific GPU devices, set the CUDA_VISIBLE_DEVICES environment variable (e.g. CUDA_VISIBLE_DEVICES=0) before launching Python or container runtimes. On multi-GPU servers or workstations containing multiple NVIDIA graphics cards, use the CUDA_VISIBLE_DEVICES environment variable to isolate execution to specific physical GPUs: Inside Docker containers, pass device indices using the --gpus container runtime flag:"
    },
    {
      "question": "Do I need to install CUDA Toolkit separately if I use PyTorch?",
      "answer": "No. Official PyTorch wheels installed via --index-url https://download.pytorch.org/whl/cu124 bundle their own isolated CUDA runtime libraries. However, installing system CUDA Toolkit (cuda-toolkit) is required if compiling custom C++/CUDA extensions from source."
    },
    {
      "question": "What flag passes specific GPUs into Docker containers?",
      "answer": "To pass all GPUs: --gpus all. To pass a specific GPU index (e.g. GPU 0): --gpus '\"device=0\"'."
    },
    {
      "question": "How do I monitor GPU memory usage in real time?",
      "answer": "Run nvidia-smi -l 1 in terminal to refresh GPU VRAM and compute utilization every 1 second, or install nvtop (sudo apt install nvtop) for an interactive graph view. ---"
    }
  ],
  "semanticFactualBody": "Leveraging NVIDIA GPU hardware acceleration is critical for modern machine learning, deep learning model training, LLM inference (Ollama, vLLM, TensorRT-LLM), and computer vision pipelines. To run GPU-accelerated AI workloads inside Docker containers or native Python environments on Ubuntu 24.04 LTS (Noble Numbat) or 22.04 LTS (Jammy Jellyfish), developers must configure three core layers: NVIDIA proprietary display drivers, the NVIDIA Container Toolkit (nvidia-ctk), and PyTorch compiled with CUDA 12 support. Here is the complete step-by-step setup guide for 2026. *Last updated: July 24, 2026* --- What are the system requirements for PyTorch CUDA hardware acceleration? Running PyTorch with CUDA hardware acceleration requires an Ubuntu or Debian Linux system with an active NVIDIA GPU, proprietary NVIDIA display drivers, and Docker Engine 24.0+ installed. Ensure your workstation or cloud server satisfies these baseline hardware and software prerequisites: **Operating System:** Ubuntu 24.04 LTS, 22.04 LTS, or Debian 12 **GPU Hardware:** NVIDIA GPU (GeForce RTX series, GTX 10 series or newer, Tesla/Data Center A100/H100/L40S) **Privileges:** User account with sudo administrative rights **Prerequisites:** Docker Engine 24.0+ and Python 3.10+ installed --- How do you install NVIDIA GPU drivers on Ubuntu? To install NVIDIA GPU drivers on Ubuntu, run ubuntu-drivers devices to inspect recommended driver packages and install nvidia-driver-550 or higher using the apt package manager. Be"
}