How to use Docker: a practical introduction.

"It works on my machine" is where machine learning projects go to die. The model runs beautifully in your environment — with your Python version, your library versions, your system packages — and falls over the moment anyone else, or any server, tries to run it. Docker solves this by packaging your code together with its entire environment into a unit that runs identically everywhere. This guide covers the mental model and the handful of commands that account for 95% of day-to-day use.

The mental model: images and containers

Docker has exactly two core nouns, and the relationship between them explains everything else:

• An image is a frozen snapshot of a filesystem plus metadata: an OS base layer, your installed packages, your code, and the command to run. Images are immutable and shareable — think of one as a class, or a recipe.
• A container is a running instance of an image — the object made from the class. You can start many containers from one image; each gets its own isolated filesystem, processes, and network. Stop it, delete it, start a fresh one: the image is untouched.

Unlike a virtual machine, a container doesn't emulate hardware or boot its own OS kernel — it shares the host's kernel and isolates at the process level. That's why containers start in milliseconds and add near-zero runtime overhead, which is exactly what you want when serving a model.

Installing: on Windows and macOS, install Docker Desktop (on Windows it runs on WSL2). On Linux servers, install Docker Engine from Docker's apt/yum repositories. Verify with:

docker run hello-world

The commands you'll actually use

docker pull python:3.12-slim      # download an image from Docker Hub
docker images                     # list images on this machine
docker run -it python:3.12-slim  bash   # start a container, interactive shell
docker ps                         # list running containers
docker ps -a                      # ...including stopped ones
docker logs -f <name>             # follow a container's output
docker exec -it <name> bash       # open a shell inside a running container
docker stop <name>                # stop
docker rm <name>                  # remove a stopped container
docker rmi <image>                # remove an image
docker system prune               # clean up stopped containers & dangling layers

Two docker run flags do most of the heavy lifting: -it gives you an interactive terminal, and --rm auto-deletes the container when it exits — ideal for experiments so you don't accumulate dead containers.

Writing a Dockerfile

A Dockerfile is the recipe that builds an image. Here is a realistic one for a Python ML service — say, a FastAPI app serving model predictions:

# Start from an official slim Python base image
FROM python:3.12-slim

# Set the working directory inside the image
WORKDIR /app

# Copy ONLY the requirements first, then install.
# This layer is cached and only re-runs when requirements.txt changes.
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Now copy the rest of the code (changes often, so it comes last)
COPY . .

# Document the port the app listens on
EXPOSE 8000

# The command the container runs on start
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Build it and run it:

docker build -t my-model-api .
docker run --rm -p 8000:8000 my-model-api

-p 8000:8000 maps port 8000 on your machine to port 8000 in the container (host:container) — the service is now at http://localhost:8000, exactly as it will be on any other machine that runs this image.

The ordering in that Dockerfile is deliberate. Docker builds images in layers and caches each one; a layer is rebuilt only if it (or a layer above it) changed. Copying requirements.txt and installing dependencies before copying your code means editing a Python file doesn't trigger a ten-minute reinstall of PyTorch — the dependency layer replays from cache. This one habit saves more build time than any other Docker trick.

Also add a .dockerignore file (same syntax as .gitignore) listing .git, __pycache__, datasets, and model checkpoints you don't want baked into the image — it keeps builds fast and images small.

Volumes: data that outlives the container

A container's filesystem is ephemeral — delete the container and everything written inside it is gone. That's a feature for reproducibility and a disaster for datasets and model outputs. The fix is to mount storage from the host:

# Bind mount: a host folder appears inside the container
docker run --rm -v /path/to/data:/app/data my-model-api

# Named volume: Docker-managed storage, survives container removal
docker volume create model-cache
docker run --rm -v model-cache:/root/.cache/huggingface my-model-api

Bind mounts are ideal during development (edit code on the host, run it in the container); named volumes are better for caches and databases. That second example is a favorite of mine: mounting the Hugging Face cache means containers don't re-download multi-gigabyte model weights on every run.

Docker Compose: multi-container setups in one file

Real projects rarely stay single-container: the API needs a database, or a vector store, or a worker. Docker Compose declares the whole stack in one YAML file. Save this as compose.yaml:

services:
  api:
    build: .
    ports:
      - "8000:8000"
    volumes:
      - ./data:/app/data
    environment:
      - DATABASE_URL=postgresql://postgres:postgres@db:5432/app
    depends_on:
      - db

  db:
    image: postgres:16
    environment:
      - POSTGRES_PASSWORD=postgres
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

docker compose up --build    # build and start everything
docker compose down          # stop and remove it all

Note the networking: the API reaches the database at the hostname db — Compose gives every service a DNS name matching its key in the file. One command now brings up the entire stack on any machine, which is transformative for onboarding collaborators.

Using GPUs in containers

Containers can use NVIDIA GPUs at full speed, and this is how most ML training and inference is deployed in practice. On a Linux host (or WSL2) with the NVIDIA driver installed, install the NVIDIA Container Toolkit once, then pass --gpus all:

# Verify GPU access from inside a container
docker run --rm --gpus all nvidia/cuda:12.6.2-base-ubuntu24.04 nvidia-smi

# Run your training image on the GPU
docker run --rm --gpus all -v ./data:/app/data my-training-image

A convenient property (covered in more depth in the CUDA post): the container brings its own CUDA runtime, and the host only needs the driver. Official images like pytorch/pytorch come with PyTorch and CUDA preinstalled, so a pinned image gives you a fully reproducible training environment — same CUDA, same libraries, every run, on any machine with an NVIDIA card.

Habits worth adopting early

• Pin your versions — base images (python:3.12-slim, not python:latest) and requirements. Unpinned builds drift and break silently.
• Prefer slim base images — smaller images build, push, and start faster; install only what the app needs.
• Pass secrets via the environment (-e API_KEY=... or an env_file), never bake them into the image — anyone with the image can read every layer.
• One process per container — API and database in separate containers, composed together, not one mega-container.
• Clean up periodically — docker system df shows what's eating disk; docker system prune reclaims it.

All posts