Free Technology learning guide
Docker Containers Basics for Developers: A Hands-On Guide
Docker Containers Basics for Developers: A Hands-On Guide — a free intermediate-level guide covering docker containers basics for developers. Learn...
What you will learn
- Introduction to Containers and Docker
- Installing and Configuring Docker
- Working with Docker Images
- Running and Managing Containers
- Dockerfile Basics
- Container Networking and Port Mapping
- Persistent Data with Volumes
- Docker Compose for Multi-Container Applications
- Docker Security Best Practices
- Debugging and Troubleshooting Docker
- Integrating Docker with CI/CD Pipelines
1. Introduction to Containers and Docker
Why “It Works on My Machine” Still Haunts Developers Imagine you’re about to ship a new feature to production. The code runs perfectly on your laptop, the CI server builds a clean image, and the QA team validates it in a staging environment. Yet, when the service is deployed to the production cluster, it crashes with cryptic errors about missing libraries and mismatched runtime versions. You spend hours digging through log files, only to discover that the production host uses a different Linux distribution, a newer version of the language runtime, and a conflicting version of a native dependency. This scenario—the classic “it works on my machine”—is the very problem containers were designed to eliminate. By packaging an application and everything it needs to run into a lightweight, portable unit, Docker lets developers define exactly what “my machine” looks like, and then ship that definition wherever it’s needed. --- 1. Containers vs. Virtual Machines: A Precise Comparison 1.1 What Is a Container? A container is an isolated runtime environment that shares the host operating system’s kernel but provides its own filesystem, process space, network interfaces, and environment variables. In practice, a container is built from a Docker image—a layered snapshot of a filesystem plus metadata—that the Docker Engine expands into a running process. 1.2 What Is a Virtual Machine? A virtual machine (VM) emulates an entire hardware stack, including its own kernel, on top of a hypervisor. Each VM runs a full guest operating system, which in turn runs the application and its dependencies. 1.3 Side‑by‑Side Comparison | Aspect | Containers | Virtual Machines | |--------|------------|------------------| | Kernel | Share host kernel | Own kernel (guest OS) | | Boot Time | Milliseconds (process start) | Seconds to minutes (full OS boot) | | Resource Overhead | Minimal (just the app + libs) | Heavy (full OS + hypervisor) | | Size | Typically tens of MB | Often GBs per VM | | Portability | OS‑level (Linux/Windows) but not across kernels | Can run any OS on any host with compatible hypervisor | | Isolation | Namespaces & cgroups (process, network, filesystem) | Full hardware isolation (CPU, memory, devices) | | Use Cases | Microservices, dev‑ops pipelines, CI/CD | Legacy monoliths, multi‑tenant workloads needing strong isolation | Bottom line: Containers give you the speed and efficiency of a process, while still offering a degree of isolation sufficient for most modern application workloads. VMs provide stronger isolation at the cost of performance and size. --- 2. The Value Proposition: Why Docker Is a Game‑Changer for Developers Docker is the de‑facto standard platform for building, shipping, and running containers. Its popularity stems from a set of concrete benefits that directly address …
2. Installing and Configuring Docker
A Real‑World Prompt to Get You Started You’ve just received a ticket from a product team: “We need a reproducible environment for the new data‑ingestion microservice. The devs want to spin it up on their laptops and push the same image to our staging cluster.” The fastest way to guarantee that every developer’s machine behaves like the staging servers is to give them a local Docker engine. If Docker isn’t installed or is mis‑configured, the team will waste hours chasing “works on my machine” bugs. This chapter gets you from a blank workstation to a ready‑to‑run Docker daemon, with the settings you need for speed, stability, and security. --- 1. Prerequisites and System Checks | Requirement | Why it matters | How to verify | |-------------|----------------|----------------| | Supported OS version | Docker Engine has minimum kernel / OS versions. | uname -r (Linux) <br swvers (macOS) <br “About” → “OS Build” (Windows) | | 64‑bit CPU | Docker uses Linux kernel features unavailable on 32‑bit. | lscpu (Linux) <br System Information (macOS/Windows) | | Virtualization enabled | On Windows/macOS the Docker Desktop VM needs VT‑x/AMD‑V. | BIOS/UEFI settings; systeminfo (Windows) shows “Hyper-V Requirements”. | | Internet connectivity | Packages are pulled from Docker’s repositories. | ping -c 1 google.com or similar. | If any of these checks fail, upgrade the OS or enable virtualization before proceeding. --- 2. Installing Docker Engine 2.1 Linux Docker provides official packages for the major distributions. The commands below assume you have sudo privileges. 2.1.1 Ubuntu / Debian 2.1.2 Fedora / CentOS / RHEL 2.1.3 Arch Linux Tip: On most Linux distros the Docker daemon is disabled by default after installation. Enable and start it with: 2.2 macOS Docker Desktop for macOS bundles the Docker Engine, a lightweight Linux VM (via hypervisor.framework), and the Docker CLI. 1. Download – Go to the Docker Desktop for Mac page and grab the latest stable .dmg. 2. Install – Drag the Docker icon onto the Applications folder. 3. Run – Open Docker from Launchpad or Spotlight. The first launch will ask for your password to install a privileged helper. Docker Desktop automatically configures the VM, but you can adjust resources (CPU, memory, disk) from the Preferences → Resources panel. 2.3 Windows Docker Desktop works on both Windows 10/11 Pro/Education (with Hyper‑V) and Windows 10/11 Home (with WSL 2). 2.3.1 Prerequisite: Enable WSL 2 (recommended) Then install a Linux distro from the Microsoft Store (Ubuntu is a common choice) and set WSL 2 as the default: 2.3.2 Install Docker Desktop 1. Download – Grab the installer from the Docker Desktop for Windows page. 2. Run – Follow the wizard; accept the option to install WSL 2 if you haven’t …
3. Working with Docker Images
Pulling Images from Docker Hub and Beyond A common first step when experimenting with containers is to pull an image that already contains the runtime you need. Imagine you’ve just been handed a ticket: “Deploy a Flask app that talks to a Postgres database. Use the smallest possible base image.” The quickest way to start is to fetch a ready‑made image from Docker Hub, then inspect it to confirm it meets the size and security constraints. The docker pull Command python – the repository name on Docker Hub. 3.11-slim – the tag that identifies a specific build of that repository. If you omit the tag Docker defaults to latest, which often points to the most recent major version. For reproducible builds you should pin to an explicit tag (e.g., 3.11.6-slim) or even a digest: A digest guarantees that you always get the exact same image layers, regardless of any later tag changes. Pulling from Other Registries Docker Hub is the default public registry, but many organizations host their own private registries (e.g., Amazon ECR, Google Container Registry, GitHub Packages). The pull syntax is identical; you just prepend the registry’s hostname: If the registry requires authentication, run docker login first: Pulling Multiple Architectures (Multi‑Arch Images) Modern images often contain manifests for several CPU architectures (amd64, arm64, etc.). Docker automatically selects the appropriate variant based on the host’s architecture: You can inspect which architectures a manifest supports with: --- Inspecting Image Metadata and Layers Once an image is on your machine, the next question is what’s inside? Docker provides a suite of commands to answer that, letting you verify provenance, size, and layer composition before you ever run a container. Listing Local Images Typical output fields: | Repository | Tag | Image ID | Created | Size | |------------|-----|----------|---------|------| | python | 3.11-slim | 7e0f2b… | 3 weeks ago | 122 MB | The Image ID is the SHA256 hash of the image’s top layer. It uniquely identifies the image in your local storage. Detailed Inspection with docker image inspect The command returns a JSON document. Key sections to focus on: RepoTags – all the tags that reference this image locally. Created – timestamp of when the image was built. RootFS – an array of layer digests (DiffIDs). Config – environment variables, entrypoint, exposed ports, etc. You can extract individual fields using Go‑style templating: Visualizing Layers with docker history Output example: | IMAGE | CREATED | CREATED BY | SIZE | COMMENT | |-------|---------|------------|------|---------| | <missing | 3 weeks ago | /bin/sh -c apt-get update && apt-get install -y … | 45 MB | | | <missing | 3 weeks ago | /bin/sh -c (nop) CMD ["python3"] | 0 B | | …
4. Running and Managing Containers
Spin Up a Container in Seconds – The “Hello, World!” API Imagine you’re on a sprint deadline and need to verify that a new version of your REST API behaves correctly against a downstream service. The service lives in a public Docker image, but your laptop’s host OS is Windows, your CI runs on Linux, and the downstream service runs on macOS. How do you get a consistent, isolated runtime without juggling VMs, installing language runtimes, or worrying about dependency conflicts? The answer is a single docker run command. In the next few sections you’ll see how to launch containers from any image, tailor their runtime parameters, and keep tight control over their lifecycle—all without leaving the terminal. --- 1. Launching Containers from Images 1.1 The Core Syntax At its heart, docker run is a one‑liner that creates + starts a container: - IMAGE[:TAG] – The immutable artifact you built or pulled (see Working with Docker Images). - COMMAND ARG… – Overrides the image’s default entrypoint if you need a different process. If you omit COMMAND, Docker uses the image’s CMD (or ENTRYPOINT if defined). Tip: The first time you run an image that isn’t present locally, Docker automatically pulls it from the registry—no extra step required. 1.2 Running in the Foreground vs. Detached Mode | Mode | Flag | Typical Use‑Case | |------|------|------------------| | Interactive (foreground) | none or -it | Debugging, REPLs, bash sessions | | Detached (background) | -d | Production‑like services, CI jobs | 1.3 Naming Your Containers Docker assigns a random name if you don’t specify one. Giving a container a human‑readable name (--name) simplifies later commands: Now docker logs my‑api or docker exec -it my‑api /bin/bash are easy to type and remember. 1.4 Custom Runtime Configurations | Need | Flag | Example | |------|------|---------| | Port mapping – expose container ports to the host | -p HOST:CONTAINER | -p 8080:80 | | Environment variables – inject config without rebuilding | -e KEY=VAL | -e APPENV=staging -e LOGLEVEL=debug | | Resource limits – bound CPU / memory | --cpus, --memory | --cpus 1.5 --memory 500m | | Custom network – attach to a user‑defined bridge | --network | --network backendnet | | Mount a host file – pass a config file (light use of volumes) | -v /path/on/host:/path/in/container:ro | -v $(pwd)/settings.yml:/app/settings.yml:ro | | Automatic cleanup – remove container when it exits | --rm | docker run --rm busybox echo "done" | Why it matters: These flags let you replicate production settings locally without altering the image itself—exactly the “immutable artifacts + fine‑grained resource limits” advantage highlighted earlier. 1.5 A Full‑Featured Example - --restart unless-stopped ensures the container survives host reboots (covered in lifecycle management). - …
5. Dockerfile Basics
A Real‑World Prompt: “I Need a Deploy‑Ready Image in 5 Minutes” Imagine you’ve just finished a weekend hackathon project—a small Flask API that returns JSON data. Your team’s CI pipeline is already set up to run Docker containers, but the build script expects a Docker image that already contains the application code, its dependencies, and a command to start the service. You have the source code locally, but no Dockerfile yet. How do you turn that handful of files into a reproducible, portable image fast enough to keep the pipeline moving? The answer lies in a Dockerfile—the declarative recipe that tells Docker how to assemble an image layer by layer. In the next sections you’ll learn the essential instructions, see a complete example, build and test the image locally, and adopt best‑practice patterns that keep your builds lean, cache‑friendly, and secure. --- Understanding Dockerfile Syntax A Dockerfile is a plain‑text file interpreted line‑by‑line. Each line (or logical block) is a Dockerfile instruction that creates a new layer on top of the previous one. Docker caches each layer, so if the instruction and its context haven’t changed, the build can reuse the cached result—dramatically speeding up subsequent builds. Below are the core instructions you’ll use for most simple images. (Later chapters will explore more advanced directives.) | Instruction | Purpose | Typical Use | |-------------|---------|--------------| | FROM | Sets the base image for subsequent instructions. Must be the first non‑comment line. | FROM node:18-alpine | | LABEL | Adds metadata (author, version, description). | LABEL maintainer="you@example.com" | | ENV | Defines environment variables that persist in the final image. | ENV PYTHONUNBUFFERED=1 | | ARG | Declares a build‑time variable, accessible only during the docker build stage. | ARG BUILDDATE | | WORKDIR | Sets the working directory for the next instructions; creates it if missing. | WORKDIR /app | | COPY | Copies files/directories from the build context into the image. | COPY . . | | ADD | Similar to COPY but can also unpack local tar archives and fetch remote URLs. Use sparingly. | ADD src.tar.gz /app/ | | RUN | Executes a command in a new layer; usually used for installing packages or compiling code. | RUN pip install -r requirements.txt | | EXPOSE | Documents the port(s) the container intends to listen on. Does not publish the port automatically. | EXPOSE 5000 | | CMD | Provides the default command for the container. Overridden by docker run <image <cmd. | CMD ["python", "app.py"] | | ENTRYPOINT | Sets a fixed executable that always runs; can be combined with CMD for default arguments. | ENTRYPOINT ["gunicorn"] | Tip: Docker ignores lines that start with , so you …
6. Container Networking and Port Mapping
Docker’s Default Networking Model When you spin up a container with docker run … without specifying a network, Docker automatically places it on the default bridge network (bridge). This network is created by the Docker Engine at installation time and provides a lightweight, isolated L2 segment for containers on the same host. Network namespace – each container gets its own network namespace (its own set of interfaces, routing tables, and firewall rules). This isolation is the same mechanism that gives containers their “process‑level” isolation discussed earlier. Bridge driver – the default bridge is implemented by the Linux bridge driver (docker0). The host’s kernel forwards traffic between the bridge and the physical NIC, applying NAT (MASQUERADE) so containers can reach the outside world. IP addressing – Docker assigns a private IPv4 address from the 172.17.0.0/16 pool (unless overridden). Containers can reach each other directly via these IPs, but the bridge does not provide automatic DNS resolution for container names unless you create a user‑defined bridge (see below). Because the default bridge is a shared network, any container attached to it can talk to any other container on the same host, but they are not reachable from the host unless you expose a port (next section). This “default‑only‑when‑you‑need‑it” model keeps the surface area small while still allowing quick ad‑hoc testing. The output shows the subnet, gateway, and a list of connected containers. If you run docker ps, you’ll notice that containers created without --network are already listed under the bridge network. --- Working with Bridge Networks While the default bridge works for simple experiments, production‑grade services benefit from user‑defined bridge networks. These give you: Custom subnets – avoid collisions with existing network ranges. Built‑in DNS – containers resolve each other by name without needing to know IPs. Isolation – containers on different bridge networks cannot communicate unless you explicitly connect them. Creating a User‑Defined Bridge Attaching Containers Now api can reach worker simply with curl http://worker:8080/…. The DNS resolution is provided by Docker’s embedded DNS server, a feature highlighted in the “Running and Managing Containers” chapter when we discussed container discovery. Inspecting the Network Look for the Containers block – it lists each container’s name, IPv4 address, and MAC. This is the first place to verify that a container received the expected IP. --- Port Mapping Basics Exposing a service to the host (and ultimately to the outside world) is done through port publishing. Docker rewrites the host’s iptables rules so that inbound traffic on a host port is forwarded to the container’s port. The -p Flag Host port – 8080 on the Docker host. Container port – 3000 inside the container (the port the app is listening on). When a client …
7. Persistent Data with Volumes
A Real‑World Pain Point: Losing Data on Container Restarts Imagine you’ve built a simple note‑taking web service. You spin up a container from an image you just built, add a few notes, and then decide to upgrade the app. You stop the old container, rebuild the image with a tiny bug fix, and start a new one. Suddenly, all the notes you entered are gone. Why? The container’s filesystem is ephemeral—when the container stops, any data written inside it disappears unless you explicitly preserve it. This chapter shows how to make your data survive container lifecycles by using Docker volumes and bind mounts. --- 1. Storage Options in Docker | Storage type | Where it lives | Managed by Docker? | Typical use‑case | |--------------|----------------|--------------------|-----------------| | Anonymous volume | Docker‑managed directory (/var/lib/docker/volumes/...) | Yes | Quick, throw‑away persistence | | Named volume | Docker‑managed directory, identified by a name | Yes | Shared data between containers, easy backup | | Bind mount | Any directory on the host filesystem | No (host‑controlled) | Development workflows, legacy data, logs | Key distinction – Volumes are Docker’s abstraction over storage; bind mounts are direct host‑directory mappings. Both solve the “data disappears” problem, but they differ in management, portability, and security. --- 2. Creating and Managing Docker Volumes 2.1. Creating a Named Volume Docker stores the volume under /var/lib/docker/volumes/notesdata/data. You can inspect it: The output shows the mountpoint, driver (local by default), and any labels. 2.2. Listing All Volumes 2.3. Removing Unused Volumes Tip: Combine with --filter to target specific volumes, e.g., docker volume prune --filter label=project=notes. --- 3. Using Volumes with Containers 3.1. Mounting a Volume at Run Time - -v notesdata:/app/data tells Docker to mount the named volume notesdata into the container at /app/data. - The container can now read/write files in /app/data, and those files persist after the container stops. 3.2. Declaring a Volume in a Dockerfile If the application expects a specific directory for persistent data, declare it in the image: When you build the image (docker build -t mynotes:latest .), the VOLUME instruction creates an anonymous volume at /app/data unless the user overrides it with -v or --mount at run time. 3.3. Sharing a Volume Between Containers Both containers see the same files under /shared. This pattern is useful for side‑car containers (e.g., a log shipper). --- 4. Bind Mounts: Direct Host Directory Access 4.1. Mounting a Host Directory - $(pwd)/data is a directory on the host (your project folder). - Changes you make locally appear instantly inside the container, and vice‑versa. 4.2. When to Prefer Bind Mounts - Rapid development: Edit source files on the host; the container picks up changes without rebuilding. - Legacy data migration: …
8. Docker Compose for Multi-Container Applications
A Real‑World Scenario: The “Shop‑Now” Stack Imagine you are tasked with prototyping a small e‑commerce site called Shop‑Now. The application consists of three loosely‑coupled components: | Service | Role | Image / Build | |---------|------|---------------| | frontend | A Node.js/Express web server that serves HTML, handles user sessions, and calls the API. | Built from frontend/Dockerfile | | api | A Python Flask REST API that talks to the database and cache. | Built from api/Dockerfile | | db | A PostgreSQL database that must persist data across restarts. | Official postgres:15 image | Individually each container can be started with docker run … (as you practiced in Running and Managing Containers), but coordinating three containers—ensuring the database is ready before the API starts, exposing the correct ports, and wiring the network so the frontend can call the API—quickly becomes a manual, error‑prone process. Docker Compose solves exactly this: a declarative YAML file that describes all the services that belong together, their build instructions, networking, volumes, and start‑up order. The rest of this chapter shows you how to turn the “Shop‑Now” stack into a single, reproducible development environment. --- What Docker Compose Adds to Your Toolkit Declarative orchestration – One file (docker-compose.yml) replaces dozens of docker run commands. Implicit networking – Compose creates a dedicated bridge network; each service is reachable by its service name as a DNS entry (no need to remember IPs). Lifecycle commands – docker compose up, down, stop, restart, logs, etc., manage the whole stack with a single command. Dependency handling – dependson and health checks let you express “service B must be healthy before service A starts”. Scalability – The same file can spin up multiple replicas of a service (--scale) without changing any code. All of these capabilities build on concepts you already know: images (from Working with Docker Images), port mapping (from Container Networking and Port Mapping), and volumes (from Persistent Data with Volumes). Compose simply packages them together. --- Anatomy of a docker-compose.yml File Below is a minimal skeleton that you will expand throughout the chapter: Key sections version – Determines which Compose file features are available; always use the latest stable (e.g., 3.9). services – The heart of the file; each key defines a container. image vs build – image pulls an existing image from a registry; build tells Compose to run docker build using the supplied context. ports – Host‑to‑container port mapping, the same syntax you saw in Container Networking and Port Mapping. environment – Inline environment variables or a reference to an .env file. volumes – Declares persistent or bind mounts; note the named volume syntax (source: db-data) that ties back to the Persistent Data with Volumes chapter. networks …
9. Docker Security Best Practices
A Breach That Started With One Bad Image Imagine a development team that ships a micro‑service every day. One night, a security analyst spots an outbound connection from a production container to an unknown IP address. The investigation traces the traffic back to a single Docker image that was built from a Dockerfile copied straight from a public GitHub repository. The image contained an outdated version of OpenSSL with a known CVE, and it ran as the root user. Within minutes the attacker gained read‑only access to the host’s file system, pivoted to other containers, and exfiltrated data. The root cause? A missing security mindset at build‑time and run‑time. This chapter equips you with the concrete steps to prevent that scenario. We’ll walk through how to: 1. Run containers with minimal privileges and non‑root users. 2. Scan images for vulnerabilities using Docker‑provided tools. 3. Harden Dockerfiles and runtime configurations using proven best practices. The material builds on the foundations laid in Dockerfile Basics and Running and Managing Containers. --- 1. Principle of Least Privilege – From Build to Run 1.1 Why “root by default” is dangerous When a container starts, Docker inherits the host’s kernel. If the process inside runs as root, it inherits all Linux capabilities that the host grants to root. Although namespaces isolate many resources, a root process can still: Escalate privileges via kernel exploits. Access host‑mounted volumes (e.g., /var/run/docker.sock). Manipulate other containers if they share the same network namespace. 1.2 Create and use a non‑root user in the Dockerfile Step‑by‑step recipe (refer to Dockerfile Basics for syntax): Why use -r? It creates a system account with no password, reducing attack surface. Avoid USER root later – any RUN statements after USER will execute as that non‑root user. 1.3 Restrict Linux capabilities at runtime Docker grants a full set of capabilities by default. Use the --cap-drop flag to remove unnecessary ones: NETBINDSERVICE lets the process bind to ports < 1024; everything else is stripped away. Combine with --cap-add only for capabilities your app truly needs. 1.4 Use read‑only root filesystem Most applications don’t need to write to the container’s filesystem. Enforce immutability: The container’s rootfs becomes read‑only; only explicitly mounted volumes can be written to. --- 2. Harden the Dockerfile – Build‑time Defenses 2.1 Base‑image hygiene Prefer official, minimal images (alpine, debian-slim). Pin versions – avoid latest tags. Example: FROM node:20.10.0-slim instead of node:latest. Check for known vulnerabilities (see Section 3). 2.2 Reduce layers and eliminate secrets Combine related commands into a single RUN to limit intermediate layers. Never store secrets (API keys, passwords) in the image. Use build‑time arguments (ARG) only for non‑sensitive data, and delete them in the same layer: After the RUN finishes, the …
10. Debugging and Troubleshooting Docker
Inspecting Running Containers When a microservice that “always worked” suddenly returns 502 Bad Gateway or the container exits with code 137, the first instinct is to reach for the Docker CLI. The commands you already use to start and stop containers become the primary lenses for diagnosing the problem. Quick status with docker ps - -a shows stopped containers, helping you spot recent crashes. - --filter narrows the list, so you’re not hunting through dozens of unrelated containers. Key columns to scan: | Column | What to look for | |--------|------------------| | STATUS | “Exited (137) 2 min ago” signals an OOM kill; “Restarting” often points to a failing health‑check. | | PORTS | Verify that the host‑to‑container mapping matches what your service expects (e.g., 0.0.0.0:8080-80/tcp). | | NAMES | Consistent naming (from the Running and Managing Containers chapter) makes correlation easier. | Deep dive with docker inspect Once you’ve identified the suspect container, docker inspect returns a JSON document with every runtime configuration: environment variables, mounted volumes, network settings, and resource limits. - [0].State – Check Running, Paused, OOMKilled, and ExitCode. - [0].HostConfig – Review CPU and memory limits you may have set in the Dockerfile Basics or via docker run. - [0].Mounts – Confirm that the source path exists on the host and that the mount mode (rw vs ro) matches expectations. A common mistake: forgetting to bind‑mount a configuration file, resulting in the container falling back to defaults and failing to start. The Mounts section reveals the mismatch instantly. Live introspection with docker top, docker stats, and docker exec | Command | Purpose | |---------|---------| | docker top <container | Shows the process tree inside the container (similar to ps). Good for spotting zombie processes or unexpected forks. | | docker stats <container | Streams live CPU, memory, network I/O, and block I/O. Spot a sudden memory spike that precedes a crash. | | docker exec -it <container sh | Drops you into an interactive shell. Use familiar Linux tools (cat, netstat, ls) to explore the runtime environment without rebuilding the image. | Tip: Combine docker exec with --user to emulate the container’s runtime user, catching permission issues that only appear under the non‑root UID you defined in Dockerfile Basics. --- Analyzing Logs and Resource Usage Even the most disciplined developer eventually needs to read the logs that Docker captures for a container. Accessing container logs - --tail limits output to recent lines, keeping the terminal readable. - --follow works like tail -f, streaming new log entries as they arrive. If the container writes to stdout/stderr (the default in most Dockerfiles), docker logs gives you the entire application output without extra configuration. Log drivers Docker supports multiple …
11. Integrating Docker with CI/CD Pipelines
The Real‑World Trigger: When a Pull Request Breaks Production Imagine a busy SaaS team that pushes a new feature every day. Yesterday a developer merged a PR that added a tiny utility library. The code compiled, the unit tests passed, but the production service crashed within minutes. Why? The new library introduced a transitive dependency that required a newer version of a native binary, and the build server had silently used an outdated base image. The incident could have been avoided if the Docker image had been built, tested, and deployed automatically before the merge reached production. That scenario is the catalyst for integrating Docker into continuous integration and continuous deployment (CI/CD) pipelines. By treating the container image as the immutable artifact—just as we learned in Working with Docker Images—the team can guarantee that every commit produces a reproducible, test‑ready environment, and that only vetted images ever touch production. --- 1. Mapping Docker into a CI/CD Workflow | Stage | Typical CI/CD Action | Docker‑specific Concern | |------|----------------------|--------------------------| | Source | Pull request, branch push | Keep Dockerfiles under version control; lint them | | Build | Compile code, run unit tests | docker build → produce an image tag that reflects the commit | | Test | Integration / smoke tests | Run containers from the built image; use Docker Compose if multiple services are needed | | Publish | Push artifacts to a repository | docker push to a registry (Docker Hub, GitHub Packages, private registry) | | Deploy | Release to staging/production | Pull the exact image tag; orchestrate via Docker Compose, Kubernetes, or a simple docker run | The pipeline mirrors the immutable artifact principle introduced earlier: each stage works with a concrete image identifier rather than a mutable filesystem. --- 2. Automating Docker Image Builds 2.1 Tagging Strategy A reliable tagging scheme makes traceability effortless: 1. Commit SHA – e.g., myapp:1a2b3c4d. 2. Branch name – e.g., myapp:feature‑login. 3. Semantic version – e.g., myapp:1.4.2. Combine them for clarity: myapp:1.4.2-feature-login-1a2b3c4d. Automated pipelines can compute these tags using environment variables supplied by the CI system. 2.2 Docker Build Command in CI --pull guarantees the latest base image (addressing the “outdated base image” risk). Labels embed provenance metadata, useful for audits and troubleshooting later. 2.3 Caching for Faster Builds Most CI platforms provide a build cache. Use Docker’s BuildKit cache export/import flags to speed up subsequent runs: Cache layers are stored in the registry, allowing parallel jobs to share them without sacrificing reproducibility. --- 3. GitHub Actions: A Hands‑On Example GitHub Actions is a native CI/CD option for projects hosted on GitHub. Below is a minimal workflow that builds, tests, and publishes a Docker image. Key points docker/build-push-action handles …
Continue learning
- Advanced Kubernetes Cluster Management for DevelopersAdvanced Kubernetes Cluster Management for Developers — a free advanced-level guide covering advanced kubernetes cluster management for developers....
- Docker for Beginners: Learn Containerization from ScratchDocker for Beginners: Learn Containerization from Scratch — a free beginner-level guide covering learn docker and containerization for beginners. Learn...
- How to Use ChatGPT for Productivity: A Beginner's RoadmapHow to Use ChatGPT for Productivity: A Beginner's Roadmap — a free beginner-level guide covering how to use chatgpt for productivity. Learn with clear...
- How to Use a 3D Printer: Beginner's Step-by-Step GuideHow to Use a 3D Printer: Beginner's Step-by-Step Guide — a free beginner-level guide covering how to use a 3d printer. Learn with clear explanations,...