Free Programming learning guide
Docker for Beginners: Learn Containerization from Scratch
Docker for Beginners: Learn Containerization from Scratch — a free beginner-level guide covering learn docker and containerization for beginners. Learn...
What you will learn
- Containers and Virtualization Fundamentals
- Installing and Verifying Docker
- Running Your First Containers
- Understanding and Managing Docker Images
- Building Custom Images with Dockerfile
- Managing Data with Volumes and Bind Mounts
- Docker Networking Basics
- Docker Compose for Multi-Container Apps
- Sharing Images with Registries
- Docker Best Practices and Optimization
- Capstone: Containerizing a Real Application
1. Containers and Virtualization Fundamentals
The "It Works on My Machine" Problem Picture this: You are a developer who has spent three weeks building a new web application. On your laptop, it runs flawlessly. You push the code to your company’s central repository, alert the operations team, and go home feeling accomplished. The next morning, the operations team tries to deploy your application to the production server. It crashes immediately. Why? Because your laptop runs a specific version of the operating system, has a particular set of system libraries installed, and relies on a specific version of a background tool called Node.js. The production server, however, runs a slightly different version of the operating system, lacks some of those libraries, and has an older version of Node.js. The operations team tells you the code is broken. You insist it works perfectly. This classic software development conflict is famously known as the "It works on my machine!" problem. For decades, the software industry struggled to bridge the gap between where code was written and where code was run. Developers and operations teams spent countless hours troubleshooting missing dependencies, mismatched configurations, and conflicting software versions. Containerization is the technology that finally solved this problem. It is a method of packaging software so that it carries everything it needs to run—code, libraries, system tools, and settings—inside a single, isolated box called a container. Because the container is self-sufficient, it runs exactly the same way on a developer's laptop as it does on a production server. Before we can understand how containers work, we need to look at the technology they evolved from: traditional virtualization. What Is Containerization? The Shipping Container Analogy To understand containerization, look to the global shipping industry. Before the 1950s, shipping cargo was a chaotic process. Goods were loaded onto ships in barrels, sacks, and wooden crates of all shapes and sizes. Loading a ship took weeks, items were frequently lost or stolen, and transferring cargo from a truck to a ship to a train required an army of workers manually repacking everything. Then came the standardized intermodal shipping container. Suddenly, a steel box of a fixed size could hold anything—electronics, clothing, or food. It didn't matter what was inside the box, because the outside was standardized. Cranes, trucks, and ships were built to handle the box itself. Shipping times plummeted, and global trade exploded. In software, containerization applies this exact same concept to code. Instead of treating an application as a messy collection of loose files, you put the application and all its dependencies into a standardized software container. The underlying server (like the ship or the truck) doesn't need to know what is inside the container. It only needs to know how to …
2. Installing and Verifying Docker
Preparing Your System for Docker In the previous chapter, we explored the fundamental differences between Virtual Machines (VMs) and containers. We learned that containers share the Host Operating System’s kernel rather than bundling a full guest OS, making them incredibly fast, efficient, and portable. But understanding the theory behind containerization is only half the battle. To actually create, run, and manage containers, you need a Container Engine running on your machine. Docker is the most popular container engine in the world. Installing it is your first practical step toward solving the classic "It works on my machine!" problem. However, because Docker interacts directly with your operating system's kernel, the installation process varies depending on whether you are using Windows, macOS, or Linux. Before diving into the installation steps, let's define a few terms you will encounter: Command Line Interface (CLI): A text-based interface where you type commands to interact with your computer. On Windows, this is Command Prompt or PowerShell; on macOS and Linux, it is the Terminal. Daemon: A background process that runs continuously, handling requests and managing tasks. Docker relies on a daemon to build, run, and distribute your containers. Here is how to install Docker based on your operating system. Installing Docker on Windows and macOS On Windows and macOS, the easiest and official way to install Docker is by using Docker Desktop. Because Docker relies on Linux kernel features to manage containers, running it natively on Windows or macOS used to be impossible. Docker Desktop solves this by automatically setting up a lightweight, hidden Linux virtual machine (using your OS's native virtualization features) to act as the Host Operating System for your containers. It then provides a graphical user interface (GUI) and a CLI that seamlessly connects to this hidden VM. Windows Installation Steps 1. Check System Requirements: Ensure you are running Windows 10 or Windows 11 (Pro, Enterprise, or Education). You will also need to ensure hardware virtualization is enabled in your computer's BIOS (often called "Intel VT-x" or "AMD-V"). 2. Enable WSL 2: Docker Desktop on Windows uses the Windows Subsystem for Linux (WSL 2) to run its background Linux VM. Open PowerShell as an Administrator and run wsl --install. You may need to restart your computer. 3. Download the Installer: Go to the official Docker website (docker.com) and download the "Docker Desktop for Windows" installer. 4. Run the Installer: Double-click the downloaded executable file. Ensure the "Use WSL 2 instead of Hyper-V" option is checked during installation. 5. Start Docker Desktop: Once installed, launch the application from your Start menu. You will see a whale icon appear in your system tray (bottom right of your screen). Wait for the icon to stop animating—this …
3. Running Your First Containers
The Magic of docker run Imagine you need to run a quick test using an outdated version of Node.js, but you don't want to clutter your computer with another installation. Or perhaps you need to spin up a PostgreSQL database for a few minutes of testing, only to throw it away when you're done. In traditional development, this means downloading installers, following setup wizards, and hoping nothing breaks your existing environment. With Docker, all of this takes a single command. Now that you have successfully installed and verified your Docker environment, it is time to experience the core promise of containerization: agility. In this chapter, we will move from theory to practice. You will learn how to use the Docker Command Line Interface (CLI) to download, run, manage, and interact with containers. The Anatomy of a Command The heart of interacting with Docker is the docker run command. At its most basic level, this command tells the Container Engine to find a specific packaged application (an image), create a live, running instance of it (a container), and execute it. Open your terminal or command prompt and type the following: When you press Enter, your terminal should output a message that starts with "Hello from Docker!" But what just happened under the hood? 1. Image Search: The Docker Engine checked your local system for an image called hello-world. 2. Image Pull: Because you didn't have it locally, Docker automatically went to Docker Hub (the default cloud registry where images are stored) and downloaded it. 3. Container Creation: Docker created a new container from that image. 4. Execution: Docker ran the default command inside the container, which printed the message to your screen. 5. Exit: Because the program's only job was to print a message, the container stopped running immediately after finishing. Running an Interactive Container The hello-world container is simple: it prints a message and exits. Most real-world applications, however, are interactive or run continuously. Let’s try running an image that allows us to interact with it directly. Run the following command: Here is what the flags and arguments mean: -it: This is actually two flags combined. The -i (interactive) flag keeps the Standard Input (STDIN) open even if not attached. The -t (tty) flag allocates a pseudo-terminal. Together, they allow you to interact with the container's command line. ubuntu: The name of the image you want to run. bash: The specific command you want to run inside the container once it starts. After pressing Enter, your terminal prompt will change to something like root@d9a3b2c4e5f1:/. You are now inside a fully functioning, isolated Ubuntu environment. Try running a few standard Linux commands: When you are done exploring, type exit and press …
4. Understanding and Managing Docker Images
Imagine ordering a flat-pack bookshelf. When it arrives, you get a box full of wooden planks, screws, and an instruction manual. You can’t put books on it yet—it’s just a set of parts. However, once you follow the instructions and assemble it, the bookshelf becomes a functional piece of furniture. In the Docker ecosystem, a container is the assembled, functioning bookshelf. The Docker image is the flat-pack box: a read-only bundle of files, libraries, and configuration instructions. Just as you can build a dozen identical bookshelves from the same flat-pack design, Docker uses a single image to spin up hundreds of identical containers. In previous chapters, we ran our first containers and saw firsthand how the Container Engine isolates processes. But how did Docker know exactly what files to put inside those containers? The answer lies in the images they were built from. What is a Docker Image? At its core, a Docker image is a read-only, immutable (unchangeable) template used to create containers. It contains everything an application needs to run: the underlying operating system files, software dependencies, libraries, environment variables, and the application code itself. Because images are read-only, you cannot directly modify an image once it is created. If you need to change something—like updating the version of a software library—you create a new image that includes the changes. Images and containers have a strictly one-way relationship: you use an image to create a container, but a container cannot exist without an underlying image. When you tell Docker to run a container, the engine looks up the specified image, unpacks its contents, and creates a writable layer on top of it for the container to use. The Layered Filesystem If a Docker image contains an entire operating system and an application, wouldn't it take up massive amounts of disk space, especially if you have dozens of images? This is where one of Docker’s most clever design features comes into play: the layered filesystem. Docker images are not single, monolithic files. They are built up in a series of read-only layers. Each layer represents a set of file system changes—like adding a file, deleting a file, or installing a software package. How Layers Work Think of the layered filesystem like a stack of transparent sheets of plastic. 1. The bottom sheet has the base operating system files drawn on it. 2. The next sheet has some system tools drawn on it. When you place it over the bottom sheet, you see both the OS and the tools. 3. The top sheet has your application code drawn on it. When you look down through the stack, you see a complete, unified file system. Docker handles this stacking process automatically. …
5. Building Custom Images with Dockerfile
Imagine downloading a brand-new laptop, turning it on, and instantly having your company’s application running perfectly—with the exact right version of Python, all the required system libraries, and every application dependency pre-installed. You didn't have to run an installer, you didn't have to configure a path variable, and you didn't have to troubleshoot a missing package. In earlier chapters, we learned how Containerization makes this magic possible, and we pulled pre-made images from registries to run containers. But what happens when you want to package your own application so it can be instantly deployed anywhere? To create a custom image, you need a blueprint. In Docker, that blueprint is called a Dockerfile. What is a Dockerfile? A Dockerfile is simply a plain text file that contains a list of instructions. It tells the Docker Engine exactly how to assemble your custom image. Think of it like a recipe: it lists your starting ingredients, the steps to prepare them, and how the final dish should be served. Because a Dockerfile is just text, it’s lightweight, easy to share, and can be tracked in version control alongside your application code. Let’s break down the core instructions you need to write your first Dockerfile. The Core Dockerfile Instructions There are many instructions you can use in a Dockerfile, but a handful of them will make up 90% of what you write as a beginner. FROM Every Dockerfile must begin with a FROM instruction. This tells Docker which existing image you want to use as your starting point. In Chapter 4, we learned that images are built in layers. The FROM instruction specifies the base layer. If you are building a Python application, you don't need to install an operating system from scratch and then install Python—you just start FROM an image that already has Python installed. WORKDIR The WORKDIR instruction sets the default working directory for any subsequent instructions. Think of it as using the cd (change directory) command inside the container. If the directory doesn't exist, Docker creates it automatically. By setting a WORKDIR, you keep your application files organized rather than scattering them across the container's root directory. COPY The COPY instruction takes files from your local computer (the host) and copies them into the container's file system. It takes two paths: the source (on your host) and the destination (inside the container). In this example, the first . means "everything in my current local directory," and the second . means "put it in the current working directory inside the container" (which we just set to /app using WORKDIR). RUN The RUN instruction tells Docker to execute a command inside the container while it is building the image. This is typically …
6. Managing Data with Volumes and Bind Mounts
The Ephemeral Nature of Containers Imagine you have just deployed a brand-new container running a PostgreSQL database. Users connect, create accounts, and save hours of data. Everything works perfectly. Then, a sudden power surge hits your server, or you simply need to stop the container to apply an update. When you start the container back up, you log in to check the database. It’s completely empty. All the user accounts and hours of data have vanished into thin air. This terrifying scenario happens because, by default, containers are ephemeral (meaning they are designed to be temporary and short-lived). When a container is stopped and removed, everything inside its writable layer is destroyed along with it. In Chapter 3, "Running Your First Containers," we explored how containers spin up from images. In Chapter 4, "Understanding and Managing Docker Images," we learned that images are made of read-only layers. When you run a container, Docker adds a thin, temporary writable layer on top of those read-only image layers. Any files the container creates, modifies, or deletes are written to this single layer. This design is fantastic for Portability and Consistency—two of the core advantages of containerization we discussed in Chapter 1. If a container is stateless (doesn't save data), you can spin up a thousand identical copies, destroy them, and replace them without a second thought. But when an application needs to remember things—like a database storing user records, a web server keeping access logs, or a game server saving player progress—this ephemeral design creates a massive problem. To solve this, Docker provides two primary mechanisms for persisting data outside the container's short lifespan: Volumes and Bind Mounts. Docker Volumes: The Managed Storage Solution A Volume is a dedicated storage space managed entirely by Docker. You can think of a volume as a USB drive that you plug into a container. The container can read and write files to the USB drive. If you unplug and throw away the container (stop and remove it), the USB drive retains all its data. You can later take that same USB drive and plug it into a completely new container. Unlike the writable layer of a container, volumes are completely independent of the container's lifecycle. They are stored in a specific directory on the Host Operating System (usually under /var/lib/docker/volumes/ on Linux), but managed exclusively by the Docker Engine. Creating and Managing Named Volumes The most common and robust way to use volumes is through Named Volumes. As the name suggests, you give the volume a human-readable name, making it easy to track. Let’s look at a practical example. Suppose you want to run a PostgreSQL database and ensure the data survives even if the …
7. Docker Networking Basics
The "Isolation Paradox" In earlier chapters, we praised containerization for its strict Isolation. We celebrated the fact that a container runs in its own environment, blissfully unaware of the host operating system or other containers. This isolation is exactly what prevents the infamous "It works on my machine!" problem. But isolation creates a paradox: if containers are completely sealed off from the outside world, how does a user send a web request to a containerized web server? And how does that web server talk to a containerized database if they are both locked in their own separate environments? The answer is Docker Networking. Just as a physical office needs doors, hallways, and internet cables for employees to communicate, containers need virtual networks to talk to each other and the outside world. The Language of Networking: Ports and IPs Before diving into Docker networks, we need to define two fundamental networking terms. Every computer on a network has an IP Address, which acts like a street address. When mail arrives at a large apartment building (the IP address), the mail carrier uses the apartment number to deliver it to the right person. In networking, that apartment number is a Port. A single computer might run a web server on port 80 and a database on port 5432. When traffic arrives at the computer's IP address labeled for port 80, the operating system knows to send it to the web server. When you run a container, it runs in an isolated environment with its own internal IP address and ports. By default, those ports are completely hidden from the host operating system. The Three Default Docker Networks When you installed Docker (as covered in Module 2), it silently created three default networks. You can see them by running the following command in your terminal: You should see three networks listed: bridge, host, and none. Let's explore what each one does. 1. The Bridge Network (The Default) The bridge network is Docker's default. Think of it as a private virtual street inside your computer. When you start a container without specifying a network, Docker automatically connects it to this bridge network. The bridge network gives your container an internal IP address, allowing it to connect to the internet (to download packages, for example). However, no one outside the host computer—including you, sitting at your keyboard—can reach the container. It is shielded from incoming traffic by default. 2. The Host Network (No Walls) The host network removes the isolation between the container and the host operating system. When you run a container on the host network, the container does not get its own internal IP address. Instead, it shares the exact same networking space …
8. Docker Compose for Multi-Container Apps
The Problem with Long Docker Commands Imagine you are building a modern blog application. It has three components: a frontend web server, a backend API, and a database. Based on what you learned in Docker Networking Basics, you know you need to create a custom network so these containers can talk to each other. You also need to set up a volume for your database so you don't lose your data when the container stops, a concept we covered in Managing Data with Volumes and Bind Mounts. To get this application running using the standard Docker CLI, your workflow would look something like this: 1. Create the network: docker network create blog-network 2. Create the database volume: docker volume create db-data 3. Start the database container: docker run -d --name db --network blog-network -v db-data:/var/lib/postgresql/data -e POSTGRESPASSWORD=secret postgres:15 4. Build your custom API image using a Dockerfile. 5. Start the API container: docker run -d --name api --network blog-network -p 8000:8000 my-api-image 6. Start the frontend container: docker run -d --name web --network blog-network -p 80:80 my-web-image If you need to tear this down, you have to stop and remove each container individually, and then decide if you want to delete the network and volume. If a colleague joins your team, you have to hand them this exact list of commands and hope they execute them in the right order. If you mistype a flag, the containers won't connect properly. This approach works for a single container, but it quickly becomes an unmanageable headache for multi-container applications. You might find yourself writing massive bash scripts just to start your local development environment. This is the exact problem Docker Compose solves. What is Docker Compose? Docker Compose is a tool designed for defining and running multi-container Docker applications. Instead of typing long, complex docker run commands in the terminal, you use a single configuration file to describe your entire application architecture. Once defined, you can start every container, network, and volume with a single command. Plain Docker vs. Docker Compose You might wonder when to use plain Docker commands versus Docker Compose. Use plain Docker commands when you are working with a single container, doing a quick test of an image, or inspecting something on the fly. Use Docker Compose when: - Your application requires more than one container (e.g., a web server and a database). - You want to version-control your container configuration alongside your application code. - You need to share your local development setup with a team. - You want to start, stop, and rebuild your entire application stack with one command. The docker-compose.yml File The heart of Docker Compose is the docker-compose.yml file. This is a text …
9. Sharing Images with Registries
The Problem with "Local Only" Images Imagine you just spent the afternoon writing a flawless Dockerfile for a new web application. You built the image, ran it locally, and it works perfectly. Now, you need to deploy it to a cloud server so the world can see it. How do you get the image from your laptop into the cloud? In Chapter 4, "Understanding and Managing Docker Images," we learned that images are built and stored locally on your machine inside the Docker engine. But if you log into a remote server and type docker run my-web-app, Docker will look at its own local storage, find nothing, and throw an error. To solve this, we use a Registry. A registry is a centralized server-side application that stores, manages, and distributes Docker images. It acts as a giant library or app store for your containers. By pushing your image to a registry, you make it available to yourself on other machines, to your teammates, or to the general public. What is a Registry? In Chapter 3, "Running Your First Containers," you already used a registry without realizing it. When you ran a command like docker run nginx, Docker reached out over the internet to Docker Hub, downloaded the nginx image, and ran it. To understand how sharing images works, we need to define a few key terms: - Registry: The actual service or server that stores images. Docker Hub is a registry, but you can also host your own. - Repository (Repo): A collection of related images with the same name but different versions. For example, the nginx repository on Docker Hub contains dozens of different versions of the Nginx web server. - Tag: A specific label applied to an image in a repository, usually denoting a version or variant. Examples include 1.23, latest, or alpine. Think of a registry as a library, a repository as a specific book title, and tags as different editions of that book. Public vs. Private Repositories When you store images in a registry, you must decide who is allowed to access them. This is governed by whether the repository is public or private. - Public Repositories: Anyone on the internet can search for and pull the image. This is ideal for open-source software (like the official ubuntu or node images) or if you want to share a project with the world. - Private Repositories: Only authenticated users (usually you, or teammates you explicitly grant access to) can pull the image. This is essential for proprietary company software, internal tools, or anything containing sensitive configurations. Most cloud-hosted registries, like Docker Hub, offer unlimited public repositories for free but charge a fee if you need multiple private …
10. Docker Best Practices and Optimization
The Cost of a Bloated Image Imagine you’ve just built a Docker image for a simple Python web API. It works perfectly on your machine. You push it to your registry, and your deployment server begins pulling the image. But instead of taking a few seconds, the download crawls. The image is 1.2 Gigabytes. For a simple web API, this is massive. Why does this happen? When developers first learn to build custom images, the focus is usually on making it work. You start with a full operating system, install every tool you might need, copy your code, and run it. However, moving from a working image to an optimized image is what separates a beginner from a proficient Docker user. A bloated image slows down your builds, eats up bandwidth when sharing images with registries, and increases the attack surface for hackers. In this chapter, we will explore how to write Dockerfiles that produce smaller, faster, and more secure images by following established industry best practices. Choosing Minimal Base Images In Chapter 5, "Building Custom Images with Dockerfile," we used the FROM instruction to specify our starting point. If you are building a Node.js application, it is tempting to use FROM node. However, the default node image is based on a full Debian Linux distribution. It includes system utilities, libraries, and files that your application doesn't actually need to run. To reduce your image size from the very first line of your Dockerfile, you should choose a minimal base image. These are stripped-down versions of operating systems or language environments that contain only the bare essentials. Slim Variants Many official Docker images offer a slim variant. A slim image is usually based on a minimal version of Debian. The maintainers have removed non-essential packages, documentation, and man pages. For example, instead of FROM node:18, you can use FROM node:18-slim. This single change can reduce your base image size by hundreds of megabytes. Alpine Linux Another popular choice is Alpine Linux. Alpine is a security-oriented, lightweight Linux distribution. A standard Ubuntu base image might be around 70MB, while an Alpine image is typically around 5MB. Many official images provide Alpine variants, denoted with the -alpine suffix (e.g., FROM node:18-alpine or FROM python:3.9-alpine). While Alpine is incredibly small, there is a trade-off to be aware of. Alpine uses a different C library (called musl) compared to standard Linux distributions (which use glibc). Most of the time, this doesn't matter. But if your application relies on pre-compiled binary packages (like some older database drivers or complex C-extensions for Python/Node), they might not work out-of-the-box on Alpine without extra configuration. Best Practice: Start with a -slim variant. If you need to squeeze out …
11. Capstone: Containerizing a Real Application
The Scenario: A Real-World Development Handoff Imagine you just finished building a brand-new web application. It has a sleek front-end interface, a back-end API that processes user requests, and a database that stores all the user data. On your laptop, it runs flawlessly. But when you hand the code over to a colleague, everything falls apart. They are running a different version of your back-end language, their database isn't configured the same way, and the app crashes on startup. They look at you and say the classic developer phrase: "It works on your machine, but not mine." Throughout the previous modules, we have assembled the tools to solve this exact problem. You learned how Containerization provides Consistency, Isolation, and Portability. You learned how to write a Dockerfile to build custom images, how to use Volumes to persist data, how to configure Docker Networking so containers can talk to each other, and how to use Docker Compose to orchestrate multi-container setups. Now, it is time to put all those pieces together. We are going to containerize a complete, multi-service application from scratch. By the end of this capstone, you will have a project that any developer can run with a single command. Meet the Application: Task Tracker Our application is a simple Task Tracker. It consists of three main components: 1. The Web Front-end: A static HTML page where users view and add tasks. 2. The Back-end API: A Python application that handles the business logic. We will use Flask, a lightweight web framework. 3. The Database: A PostgreSQL database that permanently stores the tasks. In a traditional setup, a developer would need to install Python, set up a virtual environment, install PostgreSQL locally, configure database credentials, and run both the database and the Python app simultaneously. We are going to bypass all of that by defining the entire environment as code. Here is what the project directory will look like when we are finished: Let’s build it layer by layer. Step 1: Containerizing the Back-end API The core of our application is the back-end API. It serves the front-end files and interacts with the database. First, let's look at the application code. In a real project, this would be a larger codebase, but for our purposes, a single Python file (app.py) demonstrates the concepts perfectly. backend/app.py Notice how the database credentials are not hardcoded. The app uses os.environ.get() to pull configuration from its environment. This is a crucial Docker Best Practice we covered earlier, allowing us to inject these values dynamically when the container starts. Next, we need to declare our Python dependencies. backend/requirements.txt Writing the Dockerfile Now, we write the Dockerfile for this back-end application. As we learned in …
Continue learning
- Build A Self-Hosted AI Home Server With Local LLMs And DockerBuild A Self-Hosted AI Home Server With Local LLMs And Docker — a free intermediate-level guide covering build a self-hosted ai home server with local...
- Docker Containers Basics for Developers: A Hands-On GuideDocker Containers Basics for Developers: A Hands-On Guide — a free intermediate-level guide covering docker containers basics for developers. Learn...
- Intermediate Python Automation Scripts for BeginnersIntermediate Python Automation Scripts for Beginners — a free intermediate-level guide covering intermediate python automation scripts for beginners....
- Intermediate Python Projects for Portfolio BuildingIntermediate Python Projects for Portfolio Building — a free intermediate-level guide covering intermediate python projects for portfolio building....