Docker Deep Dive

Containers vs Virtual Machines

Virtual Machines:                    Containers:
┌─────────────────────┐              ┌─────────────────────┐
│   App A   │ App B   │              │ App A │ App B │App C│
├───────────┼─────────┤              ├─────────────────────┤
│   Bins/Libs         │              │    Bins/Libs        │
├─────────────────────┤              ├─────────────────────┤
│  Guest OS │Guest OS │              │  Container Runtime  │
├─────────────────────┤              ├─────────────────────┤
│     Hypervisor      │              │      Host OS        │
├─────────────────────┤              ├─────────────────────┤
│      Host OS        │              │      Hardware       │
├─────────────────────┤              └─────────────────────┘
│      Hardware       │
└─────────────────────┘

Linux Kernel Features

1. Namespaces (Isolation)

Isolate what process can see:

# See container namespaces
docker inspect --format '{{.State.Pid}}' container_name
ls -la /proc/PID/ns/

# Example output:
lrwxrwxrwx 1 root root 0 net:[4026532208]
lrwxrwxrwx 1 root root 0 pid:[4026532209]
lrwxrwxrwx 1 root root 0 mnt:[4026532206]
...

2. Control Groups (cgroups) - Resource Limits

Control what resources process can use:

# Run container with resource limits
docker run -d \
  --name myapp \
  --cpus=2 \               # 2 CPU cores
  --memory=1g \            # 1GB RAM
  --memory-swap=2g \       # 2GB total (RAM + swap)
  --pids-limit=100 \       # Max 100 processes
  nginx

# View resource usage
docker stats myapp

3. Union Filesystems (Layers)

Stack read-only layers + writable top layer:

Container Filesystem (OverlayFS):

┌─────────────────────────────────┐
│  Container Layer (R/W)          │  ← Changes here
│  /tmp/myfile.txt                │
├─────────────────────────────────┤
│  Image Layer 3 (R/O)            │
│  ADD app.jar /app/              │
├─────────────────────────────────┤
│  Image Layer 2 (R/O)            │
│  RUN apt-get install openjdk    │
├─────────────────────────────────┤
│  Image Layer 1 (R/O)            │
│  FROM ubuntu:22.04              │
└─────────────────────────────────┘

Copy-on-Write (CoW): Modify file in read-only layer → copy to writable layer first

What is an Image?

A read-only template with instructions for creating a container. Built from layers stacked on top of each other.

Image Layers

# View image layers
docker history nginx:latest

IMAGE          CREATED BY                                      SIZE
f9c14fe76a38   /bin/sh -c #(nop) CMD ["nginx" "-g" "daemon…   0B
      /bin/sh -c #(nop) EXPOSE 80                     0B
      /bin/sh -c apt-get update && apt-get install…   54MB
      /bin/sh -c #(nop) WORKDIR /usr/share/nginx/…   0B
      /bin/sh -c #(nop) FROM debian:bookworm          124MB

Image Storage Drivers

# Check storage driver
docker info | grep "Storage Driver"
Storage Driver: overlay2

Optimize Layer Caching

FROM node:18
WORKDIR /app
COPY . .                    # Copies everything
RUN npm install             # Cache invalidated on ANY file change
CMD ["node", "server.js"]

FROM node:18
WORKDIR /app

# Copy only package files first
COPY package*.json ./
RUN npm install             # Cached unless package.json changes

# Copy code after dependencies installed
COPY . .
CMD ["node", "server.js"]

Multi-Stage Builds

Reduce final image size by using separate build and runtime stages.

FROM golang:1.21
WORKDIR /app
COPY . .
RUN go build -o myapp
CMD ["./myapp"]

# Final image: 1.2GB (includes Go compiler, build tools)

# Stage 1: Build
FROM golang:1.21 AS builder
WORKDIR /app
COPY go.* ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o myapp

# Stage 2: Runtime
FROM alpine:3.19
RUN apk --no-cache add ca-certificates
WORKDIR /app
COPY --from=builder /app/myapp .
CMD ["./myapp"]

# Final image: 15MB (only binary + Alpine)

More Multi-Stage Examples

# Python with compiled dependencies
FROM python:3.11 AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt

FROM python:3.11-slim
WORKDIR /app
COPY --from=builder /root/.local /root/.local
COPY . .
ENV PATH=/root/.local/bin:$PATH
CMD ["python", "app.py"]

# React/Node.js app
FROM node:18 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM nginx:alpine
COPY --from=builder /app/build /usr/share/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

Minimize Layers

RUN apt-get update
RUN apt-get install -y curl
RUN apt-get install -y nginx
RUN apt-get install -y vim
# 4 layers!

RUN apt-get update && apt-get install -y \
    curl \
    nginx \
    vim \
 && rm -rf /var/lib/apt/lists/*
# 1 layer, cleaned up package cache

# Good: Cleanup in same RUN command
RUN apt-get update && apt-get install -y curl \
 && rm -rf /var/lib/apt/lists/*

# Bad: Separate RUN for cleanup (doesn't reduce image size!)
RUN apt-get update && apt-get install -y curl
RUN rm -rf /var/lib/apt/lists/*  # Too late, layer already created

.dockerignore

Exclude files from build context (like .gitignore):

# .dockerignore
node_modules
npm-debug.log
.git
.env
*.md
.DS_Store
Dockerfile
.dockerignore
.vscode
.idea
coverage/
dist/
*.test.js

Use Specific Tags

FROM node:latest      # Can break unexpectedly
FROM python:3         # Ambiguous (3.9? 3.10? 3.11?)

FROM node:18.19-alpine    # Specific version
FROM python:3.11.7-slim   # Specific version
FROM nginx:1.25-alpine    # Specific version

Network Drivers

1. Bridge Network (Default)

# Create custom bridge network
docker network create mynet

# Run containers on same network
docker run -d --name web --network mynet nginx
docker run -d --name api --network mynet node:18

# Containers can communicate by name
# From 'web' container: curl http://api:3000

Host Network Namespace:
┌────────────────────────────────────┐
│ eth0 (192.168.1.100)               │
│                                    │
│  Docker Bridge (docker0)           │
│  172.17.0.1                        │
│    │                               │
│    ├─ Container 1: 172.17.0.2:80   │
│    │    nginx                      │
│    │                               │
│    └─ Container 2: 172.17.0.3:3000 │
│         node app                   │
└────────────────────────────────────┘

External → 192.168.1.100:8080 → 172.17.0.2:80 (port mapping)

# Port mapping
docker run -d -p 8080:80 nginx
# Host port 8080 → Container port 80

# Publish all exposed ports to random host ports
docker run -d -P nginx

2. Host Network

# Container uses host network stack directly
docker run -d --network host nginx

# No port mapping needed
# Container port 80 = Host port 80
# Better performance, less isolation

Container Communication

Service Discovery

# Docker provides built-in DNS
docker network create myapp
docker run -d --name db --network myapp postgres
docker run -d --name api --network myapp node:18

# From 'api' container:
# Can access 'db' by hostname:
curl http://db:5432  # Resolves to db container IP

Environment Variables

# Pass database URL to app container
docker run -d --name api --network myapp \
  -e DATABASE_URL=postgresql://db:5432/mydb \
  node:18

Storage Types

1. Volumes (Recommended)

# Create named volume
docker volume create pgdata

# Use volume
docker run -d --name postgres \
  -v pgdata:/var/lib/postgresql/data \
  postgres:15

# Volume persists after container deletion
docker rm -f postgres
docker run -d --name postgres2 \
  -v pgdata:/var/lib/postgresql/data \
  postgres:15
# Data still there!

# List volumes
docker volume ls

# Inspect volume
docker volume inspect pgdata

# Remove volume
docker volume rm pgdata

2. Bind Mounts

# Mount host directory into container
docker run -d --name web \
  -v /home/user/website:/usr/share/nginx/html:ro \
  nginx

# :ro = read-only
# Changes on host immediately visible in container

# Development example
docker run -it --rm \
  -v $(pwd):/app \
  -w /app \
  node:18 \
  npm run dev

3. tmpfs (Memory)

# Mount tmpfs (in-memory, not persistent)
docker run -d --name app \
  --tmpfs /tmp:rw,size=100m,mode=1777 \
  myapp

# Use case: Sensitive temporary files, fast I/O

Multi-Container Applications

# docker-compose.yml
version: '3.8'

services:
  web:
    image: nginx:alpine
    ports:
      - "8080:80"
    volumes:
      - ./html:/usr/share/nginx/html:ro
    networks:
      - frontend
    depends_on:
      - api

  api:
    build:
      context: ./api
      dockerfile: Dockerfile
    environment:
      - DATABASE_URL=postgresql://postgres:secret@db:5432/mydb
      - REDIS_URL=redis://cache:6379
    networks:
      - frontend
      - backend
    depends_on:
      - db
      - cache

  db:
    image: postgres:15
    environment:
      - POSTGRES_PASSWORD=secret
      - POSTGRES_DB=mydb
    volumes:
      - pgdata:/var/lib/postgresql/data
    networks:
      - backend

  cache:
    image: redis:7-alpine
    networks:
      - backend

networks:
  frontend:
  backend:

volumes:
  pgdata:

# Commands
docker compose up -d           # Start all services
docker compose ps              # List services
docker compose logs -f api     # Follow logs
docker compose exec api sh     # Shell into container
docker compose down            # Stop and remove
docker compose down -v         # Stop and remove volumes