Free Programming learning guide
Build A Self-Hosted AI Home Server With Local LLMs And Docker
Build A Self-Hosted AI Home Server With Local LLMs And Docker — a free intermediate-level guide covering build a self-hosted ai home server with local...
What you will learn
- Iron Before Ignorance: Scoring the Right Rig
- Bare Metal or Bust: OS Foundation
- Containerize or Capitulate: Docker Domination
- The Engine Room: Local LLM Runners
- Model Wrangling: Pick, Quantize, Deploy
- Face of the Beast: Frontend UIs
- Wires and Tunnels: Network Access
- Lock It Down or Get Clowned
- Data Vaults: Persistent Storage Strategy
- Give It a Memory: RAG and Vector Databases
- Pulse Check: Monitoring and Health
- Autopilot: Compose, Update, Repeat
1. Iron Before Ignorance: Scoring the Right Rig
Picture this: you just dropped $2,400 on a tricked-out gaming PC. RGB lights dancing like a damn disco. 64 gigs of RAM. Top-of-the-line processor. You fire up your first local LLM, hit enter, and watch your beautiful beast choke so hard it begs for mercy. The model spits out one word every six seconds. You could read the entire dictionary in the time it takes this thing to generate a paragraph. Welcome to the consequences of being hardware-ignorant, you dumb beautiful bastard. Pull up a chair. We're fixing your brain before you touch a single credit card. You're about to build a self-hosted AI home server. Not a gaming rig. Not a NAS that occasionally transcodes a movie. A goddamn inference engine that thinks. And if you get the iron wrong on day one? Every single chapter after this is you trying to polish a turd. I love you too much to let you do that to yourself. So we're going deep on hardware. By the end of this, you'll know exactly what to buy, what to avoid like a radioactive ex, and why your current "gaming PC" might be the worst possible starting point for this project. Core Carnage (Rip Apart the Essentials) The GPU Cult: Why Your Graphics Card Is the Only Thing That Matters Here's the truth that hurts: for running local LLMs, your CPU is a glorified paperweight. Your RAM is a backup plan. Your GPU? That's the entire ballgame. Let me explain why before you start whining about your Ryzen 9. Back in the stone age of AI (like, 2012), researchers used CPUs for everything. Then some absolute psychopaths at NVIDIA realized that the parallel processing power needed to render video game graphics was mathematically identical to the matrix multiplication needed for neural networks. A CPU has maybe 8 to 24 cores. Each one is a brilliant scholar who can solve complex problems sequentially. A GPU has thousands of cores. Each one is a dumb little worker ant that can only do basic math—but it can do thousands of calculations simultaneously. LLMs are, at their core, just massive chains of matrix multiplication. Billions of parameters multiplied against each other, token by token. ⚠️ Common Mistake: Buying a 24-core Threadripper and skipping the dedicated GPU because "more cores = more power." I've watched guys drop three grand on a CPU and then ask why a 7B model runs slower than their grandma typing with mittens. CPUs process tokens sequentially. GPUs process them in parallel. For LLMs, parallel wins. Every. Single. Time. Now, here's where it gets brutal. Not all GPUs are created equal. And I don't mean speed—I mean architecture. NVIDIA owns this space. Not because …
2. Bare Metal or Bust: OS Foundation
Picture this: it's 2 AM, your model is mid-inference, and your server decides it's a great time to download a 4-gigabyte Windows update, reboot, and lock you out because Microsoft thinks you really need to see a screensaver of a waterfall. If you're running your AI rig on Windows, you're not a server admin — you're a tourist with a very expensive space heater. You dumb beautiful bastard. You survived Chapter 1. You've got the rig, the GPU, the silicon that cost you a month's rent. But right now, that hardware is naked. Exposed. Running whatever bloated commercial OS came pre-installed on it. We're stripping it to the fcking studs today. Core Carnage (Rip Apart the Essentials) Your operating system is the bouncer between your hardware and your models. If that bouncer is busy updating Candy Crush, your model waits. For a server, that's unacceptable. The Headless Mandate A "headless" server means no GUI. No desktop. No mouse. No pretty little taskbar. Just a black terminal screen and the command line. Why? Because every cycle your CPU spends rendering a transparent window shadow is a cycle it's NOT spending on inference. A desktop environment eats 1-2 GB of RAM just sitting there, breathing. On a server, that's a sin. You run headless or you run home. ⚠️ Common Mistake: Installing Ubuntu Desktop "just in case I need a GUI later." No. Stop. You will NEVER need that GUI. SSH exists. If you need to see a web page, use the browser on your laptop. Your server is a workhorse, not a display pony. The Distro Decision: Ubuntu Server vs. Debian vs. Proxmox I know what you're thinking: "There are 500 Linux distros, how do I pick?" Cute. Here's the answer: you pick the one that has the most documentation and the largest community, because when (not if) something breaks at 3 AM, you need Stack Overflow to have the answer. Ubuntu Server LTS (Long Term Support) is the reigning champ for beginners. It's the Toyota Corolla of Linux — not exciting, but it starts every morning, parts are everywhere, and everyone knows how to fix it. LTS means 5 years of guaranteed updates. You don't upgrade your OS every 6 months on a production server. That's insane. Debian is Ubuntu's austere, slightly faster grandfather. Ubuntu is built on Debian. Debian is leaner, more stable, and has zero corporate baggage. But it ships with older packages, which means driver headaches if you have cutting-edge GPUs. Proxmox is the wild card. It's not just an OS — it's a Type-1 hypervisor. It runs directly on your bare metal and lets you carve your machine into virtual machines (VMs) and Linux Containers (LXC). …
3. Containerize or Capitulate: Docker Domination
Picture this: it's 2 AM. Your model just crashed. You SSH into your box — the one you set up in Chapter 2 like a damn champion — and you're staring at seventeen different processes, four broken Python environments, a GPU driver that decided to sit the bed during an apt upgrade, and dependencies scattered across your filesystem like shrapnel after a pipe bomb. You don't know what broke. You can't undo it. And your model? Dead. Cold. Gone. That nightmare? That's what life was like BEFORE Docker. Every. Single. Time. You beautiful bastard, you survived OS installation. You've got a bare metal box running Ubuntu Server LTS, SSH locked down, GPU drivers humming. You should be proud. Now throw all that pride in the trash because if you install your AI stack the way cavemen did — apt install this, pip install that, curl | bash like an animal — you're building a house of cards on a fault line during a fcking earthquake. Enter Docker. The container. The single most important tool in modern infrastructure. And by the end of this chapter, you're going to understand it better than 90% of "DevOps engineers" on LinkedIn who put it on their resume because they once ran docker pull hello-world. Core Carnage (Rip Apart the Essentials) Why Docker Exists (The "Works on My Machine" Plague) Here's a fun fact that'll mess with your head: Docker didn't invent containers. Not even close. Container technology has existed in Linux since 1979 with chroot. Solaris Zones came in 2004. LXC — the thing Docker was originally built on top of — landed in 2008. Solomon Hykes and his team at dotCloud (a Platform-as-a-Service company) were using LXC internally and realized they'd built something incredible on top of it. They open-sourced it in 2013 as Docker, and the tech world lost its collective mind. Why? Because Docker solved the single most expensive, soul-crushing problem in software engineering: "It works on my machine." You know the drill. Developer writes code on their MacBook. It works. They push it to a server. It explodes. Why? Because the server has a different version of libc. Because the Python path is different. Because some ashole ran apt upgrade six months ago and silently broke a dependency. Because the GPU driver on the server doesn't match the CUDA version the code expects. Every single one of these is a 4 AM phone call waiting to happen. Docker fixes this by packaging your application AND everything it needs — the OS libraries, the runtime, the dependencies, the config files — into a single, immutable artifact called an image. That image runs identically on your laptop, your server, your coworker's machine, …
4. The Engine Room: Local LLM Runners
Picture this: you've got a beautiful RTX 3090 sitting in your server. Twenty-four gigs of VRAM. You've installed Ubuntu Server LTS like a pro, Docker is running clean, containers are humming. You feel like a god. Then you try to run a model and your terminal throws some error about "GGML tensor allocation failure" and your GPU sits there at 0% utilization like a Ferrari parked in a garage with no gas. What the hell happened, champ? You forgot the engine. You built the chassis, wired the electrics, polished the paint — and never dropped the motor in. That's what this chapter is about. The inference engine. The thing that actually takes a model file and makes it TALK. Without an inference engine, your model files are just multi-gigabyte paperweights sitting on a filesystem doing absolutely nothing. Like buying a Steinway piano and not knowing how to play Chopsticks. The model is the sheet music. The engine is the pianist. And right now, you don't have a pianist. Let's fix that. Core Carnage (Rip Apart the Essentials) What the Fck Is an Inference Engine, Actually? Here's where 90% of people get it twisted. They think "local LLM" means one thing. It doesn't. There are TWO completely separate components: 1. The Model — a file (usually .gguf, .safetensors, or .bin) containing billions of numbers called "weights." These are the learned parameters. It's a frozen brain. It knows things but can't think on its own. 2. The Inference Engine — the software that loads those weights into VRAM, accepts your text prompt, runs the matrix math, and spits out tokens. It's the runtime. The animator. The thing that makes the dead brain alive. You need BOTH. And the engine you pick determines everything — speed, compatibility, API surface, how many people can talk to your model at once, and whether your GPU actually does work or just watches from the sidelines. The Holy Trinity of Local Inference There are three engines that matter right now. Not ten. Not "here's a top 10 list from some blog farm." Three. Anyone who tries to sell you on a fourth is either selling something or hasn't actually deployed in production. --- 1. Ollama — The People's Champion Ollama was built by the team behind Docker. Literally. Same DNA. And you can tell, because it does for LLMs what Docker did for deployment: it made the hard part disappear. You want to run Llama 3? ollama run llama3. That's it. It pulls the model, quantizes it automatically, loads it into VRAM, gives you a chat prompt, AND spins up an OpenAI-compatible REST API on port 11434. No config file surgery. No Python environment hell. No "install …
5. Model Wrangling: Pick, Quantize, Deploy
You've got a 24GB RTX 3090 sitting in your rig, a Docker containerization strategy locked down from the last chapter, and an inference engine ready to rip. So what do you do? You go to Hugging Face, download a 70-billion parameter model in FP16, hit run, and watch your system instantly crap itself and reboot like it just had a neurological event. Congratulations, genius. You just committed the cardinal sin of local AI. You tried to shove a watermelon through a garden hose. Welcome to Model Wrangling. This is where the wannabes get filtered out. This is where we take the theoretical horsepower of your Tier 3 monster rig and actually make it do something useful without melting a hole through your floorboards. Core Carnage (Rip Apart the Essentials) The Memory Math That Nobody Taught You Here's the deal, kid. Parameters are the "brain cells" of your model. A 7B model has 7 billion of them. A 70B model has 70 billion. Revolutionary, I know. But here's what nobody fcking explains to you until you've already wasted three hours downloading a file you can't use: every single parameter needs memory to store it. At FP16 (16-bit floating point — the "native" format most models are trained in), each parameter eats 2 bytes. That's just math. Not magic, not AI wizardry, just cold hard bytes. So let's do bar napkin math: - 7B parameters × 2 bytes = 14 GB of VRAM just to LOAD the model - 70B parameters × 2 bytes = 140 GB of VRAM You have 24 GB on that RTX 3090. You see the fcking problem? And it gets worse. You don't just need memory for the model. You need memory for the KV cache — that's the memory the model uses to remember your conversation as it generates tokens. You need memory for the context window. You need memory for the OS, for Docker, for the inference engine itself. ⚠️ Common Mistake: Thinking "I have 24GB VRAM so I can run a 24GB model." No. You can run maybe a 16-18GB model if you want any context window at all. The model is the passenger; the KV cache is the luggage. You need room for both or the car doesn't move. The rule of thumb that actually works in the real world: Total VRAM needed ≈ Model size + (Context window × Overhead factor) For a 7B model at Q4 (more on that in a second), you're looking at roughly 5-6 GB for the model plus 1-2 GB for context. That fits comfortably in 8 GB. For a 70B at Q4, you're looking at 40+ GB. That does NOT fit in 24 GB unless you …
6. Face of the Beast: Frontend UIs
Picture this: it's 2 AM, your model just spat out the most brilliant piece of code you've ever seen, and you're sitting there staring at a terminal window like a feral raccoon who just found a whole pizza in a dumpster. No syntax highlighting. No conversation history you can scroll back through without getting carpal tunnel. No way to upload a file. You're piping text into curl commands like it's 1997. That's not a workflow, you dumb beautiful bastard. That's a cry for help. You've got the engine running. You've got Docker containerizing your sht like a pro. You've got Ollama serving models and quantization squeezing every last drop of performance out of your GPU. But right now, you're driving a Ferrari through a mail slot. It's time to bolt on a real steering wheel. Core Carnage (Rip Apart the Essentials) Why Frontend UIs Exist (And Why You Should Care) Here's the thing, champ. LLM inference engines — Ollama, vLLM, llama.cpp — they don't give a damn about you. They speak one language: API calls. You send a JSON payload, they send back tokens. That's it. That's the whole conversation. The original LLM paper from 2018? Vaswani and crew didn't exactly design transformers with "user experience" in mind. They were solving a math problem — sequence-to-sequence translation with attention mechanisms. The fact that you want a pretty chat bubble interface would've sounded absurd to them. But here's the mind-blown moment you didn't see coming: the frontend you choose literally changes the quality of answers you get. Not because the model is different. Because the system prompt, the conversation formatting, the context window management, the document preprocessing — all of that happens in the UI layer before a single token hits your model. A shtty frontend feeds your model garbage context. A good frontend is like a really good translator at a UN meeting — it makes sure both sides actually understand what the hell is happening. So no, this isn't just "making it pretty." This is the last mile of inference quality. The Two Heavyweights: Open WebUI vs. LibreChat Listen up, genius. There are approximately nine thousand chat UIs floating around GitHub right now. Ninety percent of them are someone's weekend project that hasn't been updated since the model was called "text-davinci-003." We're not touching those with a ten-foot pole. We're looking at the two that actually matter: Open WebUI — Born from the Ollama community, this thing was originally called "Ollama WebUI" before they realized maybe they shouldn't tie their entire brand identity to one inference engine. It's a Python/Svelte application that talks directly to Ollama's API. Think of it as the native tongue of your local LLM setup. …
7. Wires and Tunnels: Network Access
Picture this: It's 2 AM. You're at a hotel in another state. You need to pull up your local LLM to draft something critical. You open your browser, type your server address, and... nothing. Dead air. Because your dumbass thought port forwarding was a personality trait and now some botnet in Eastern Europe is using your GPU to mine Monero while your model sits there useless. Welcome to Chapter 7, you dumb beautiful bastard. The chapter where your server stops being a lonely hermit screaming into the void and actually becomes REACHABLE — without you handing the keys to every script kiddie on the planet. You've got a beast running. Ubuntu Server is purring. Docker containers are behaving. Your LLM is generating tokens like a champ. Your frontend UI is slick. And right now, the only way to access any of it is by being physically on your network like some kind of caveman hunched over a keyboard in your basement. That ends today. Core Carnage (Rip Apart the Essentials) The Problem You Don't Know You Have Here's what most rookies do: they find their server's local IP, slap :8080 on the end, and call it a day. Then they want remote access, so they Google "how to port forward," punch a hole in their router, and expose port 8080 to the entire goddamn internet. Congratulations, genius. You just painted a target on your back the size of a billboard. ⚠️ Common Mistake: Exposing ports directly to the internet. Every exposed port is a door. Every door is a target. Automated scanners find open ports in MINUTES. Not hours. Not days. Minutes. Your little home server is getting probed right now if you did this. There are three problems with the rookie approach: Problem 1: Port Sprawl. Every service gets its own port. Your LLM runner on 8080. Your frontend on 3000. Your vector database on 6333. Your monitoring on 9090. You're memorizing port numbers like a phone book from 1995. That's not a system. That's a mess. Problem 2: Zero Encryption. HTTP is plaintext. That means everything between your browser and your server is readable by anyone in between. Your prompts, your data, your API keys — all flying through the air naked. You wouldn't write your passwords on a postcard, but that's essentially what you're doing. Problem 3: The Router Roulette. Port forwarding means trusting your router's firewall to protect you. The same router that probably hasn't had a firmware update since the Bush administration. The same router with default admin credentials that are posted on a wiki somewhere. We're fixing all three. But first, you need to understand the weapons in your arsenal. Reverse Proxy: The Bouncer …
8. Lock It Down or Get Clowned
I once watched a guy spend three months building a beautiful self-hosted AI stack — dual GPUs, 64GB RAM, a custom Open WebUI setup, the works. He exposed port 3000 to the internet with a password of "admin123" because he "just wanted to test it real quick." Forty-seven minutes. That's how long it took for a bot to find it, brute-force it, and turn his $4,000 rig into a Monero miner that ran so hot it thermal-throttled his GPUs into the stone age. He didn't notice for nine days. Nine. Days. His electric bill looked like he was running a grow house. His hardware aged a year in a week. And the worst part? He's not stupid. He was just lazy for forty-seven minutes. You dumb beautiful bastard, you have spent seven chapters building something genuinely impressive. You've got a real OS on real metal, Docker containers running actual LLMs, a slick frontend, and network access that doesn't suck. You're a goddamn warlord at this point. But warlords get stabbed in their sleep when they leave the back gate open. This chapter is about making sure nobody walks through your gate, punches your horse, and burns your village down while you're off bragging about your local AI setup on Reddit. Lock the fck in. Core Carnage (Rip Apart the Essentials) Layer 1: The Firewall — Your First Line of "Go Away" A firewall is not some magical force field. It's a bouncer. It stands at the door and checks IDs. That's it. The concept has existed since the late 1980s when the first network firewalls were developed at Digital Equipment Corporation — and the core idea hasn't changed because it doesn't need to. You either let traffic through or you don't. The golden rule of firewalls, the one rule that if you remember nothing else you remember THIS: default-deny everything, then explicitly allow only what you need. Think about it like a club. You don't open all the doors and then try to individually lock the ones you don't want people walking through. You lock EVERYTHING and then unlock the VIP entrance. One is a security strategy. The other is a panic attack waiting to happen. UFW — Uncomplicated Firewall Ubuntu ships with UFW, and it is exactly what it sounds like — a wrapper around iptables that doesn't make you want to die. iptables is powerful but reads like ancient Sanskrit translated through a broken calculator. UFW exists because someone at Canonical realized normal humans need to configure firewalls too. Here's your baseline configuration. Memorize this like your Social Security number: See what happened there? I allowed SSH from a specific IP range — your local network. Not the …
9. Data Vaults: Persistent Storage Strategy
Chapter 9: Data Vaults — Persistent Storage Strategy Picture this: It's 2 AM. Your LLM just hallucinated a conversation history so good you wanted to save it. You go to check your Docker logs, run a docker compose down to update the image, and wake up to a fresh, empty instance. Your data? Gone. Evaporated. Like a fart in a hurricane. You just nuked three weeks of conversation history because you didn't understand persistent storage. Congratulations, champ — you played yourself. I told you in Containerize or Capitulate: Docker Domination that containers are ephemeral. I told you they die. I told you they don't give a damn about your feelings. But did you listen? Did you map your volumes? Or did you just docker run your way into a nightmare? You're here because you want a real server. A server that survives updates, crashes, and your own stupidity. That means we need to talk about data persistence — the art of making sure your sht doesn't disappear when the container does. Still breathing? Good. Because this next part separates the pretenders from the players. Core Carnage (Rip Apart the Essentials) The Container Lie Listen to me very carefully, you dumb beautiful bastard: Containers are not servers. They are processes. They are temporary workers. When a container stops, everything inside it — every file, every database, every saved conversation — goes to the void. It's like writing your memoirs on a whiteboard during an earthquake. Docker solves this with volumes and bind mounts. These are your lifelines. Your umbilical cord to reality. Without them, you are a goldfish swimming in a bowl that gets smashed every Tuesday for a software update. The Two Storage Beasts Docker gives you two main ways to persist data. Fck this up, and you're a dead man walking. 1. Named Volumes (The Vault) Think of named volumes like a safety deposit box at a bank. You don't know where it is. You don't care where it is. You just hand Docker a key, and Docker hands you your sht back. That's it. Docker creates a managed volume called llmdbdata. It lives somewhere in /var/lib/docker/volumes/ on your host. You don't touch it. You don't look at it. You don't edit files in it directly. You let Docker be the bank. When to use named volumes: - Databases (Postgres, Redis, vector DBs) - Application state you don't need to manually edit - Anything that needs to survive a container nuke 🎯 Key Insight: Named volumes are managed by Docker. That means Docker handles permissions, ownership, and lifecycle. You don't fight with chown or chmod like an idiot at 3 AM. The container owns it. You respect that. 2. …
10. Give It a Memory: RAG and Vector Databases
Your LLM just told you the CEO of Nintendo is named Shigeru Miyamoto and that the company was founded in 1989. You nodded, printed the report, and handed it to your boss. Congratulations, you dumb beautiful bastard — you just committed professional suicide. An LLM without a retrieval system is the drunkest guy at the bar who swears he's a Navy SEAL. He speaks with absolute confidence. He will fight you on the facts. And he will lie to your face without breaking eye contact. You don't need a bigger model. You don't need more parameters. You need to hand the drunk bastard a cheat sheet before he opens his trap. That cheat sheet? That's RAG — Retrieval-Augmented Generation. And by the end of this chapter, your server will have the memory of an elephant and the honesty of a nun. Core Carnage (Rip Apart the Essentials) What the Hell Is RAG and Why Should You Care? Here's the problem nobody warned you about: LLMs are frozen in time. That 8B model you quantized in Module 5? It learned everything it knows from a training cutoff date. It doesn't know your company's Q3 financials. It doesn't know your wife's birthday. It doesn't know the API documentation for the library you installed last Tuesday. You could fine-tune the model. Sure. Go ahead. Spend 40 hours and $600 in compute teaching a model facts that will change next month. That's like tattooing your grocery list on your forearm. Good luck updating it when you run out of milk. RAG takes a different approach. Instead of tattooing facts onto the model, you build a search engine that sits next to it. When you ask a question, the search engine digs through your actual documents, grabs the relevant chunks, and slips them to the LLM like a student passing notes during an exam. The LLM reads the notes and answers based on those, not its own hallucinated memory. 🎯 Key Insight: RAG doesn't teach the model new facts. It gives the model a reference library to check before answering. You're not sending the model to school — you're letting it use open notes on the test. The concept was introduced by researchers at Facebook AI (Lewis et al., 2020) because even mega-corporations realized you can't keep retraining models every time a PDF updates. The paper was called "Retrieval-Augmented Generation: Knowledge-Intensive NLP Tasks for Pre-Trained Sequence-to-Sequence Models." Classic academic naming — ten words when three would do. But the idea was revolutionary: separate the knowledge from the reasoning. Your LLM handles the reasoning. Your vector database handles the knowledge. Clean separation. Easy updates. No retraining. The Three-Headed Beast: Embeddings, Vector DBs, and Chunks RAG has …
11. Pulse Check: Monitoring and Health
Your server died at 2:47 AM. You don't know this yet. You're asleep, dreaming about something pleasant — maybe a world where Docker containers restart themselves without throwing a tantrum. Meanwhile, your GPU is choking on a runaway process, your disk is at 99%, and your LLM has been returning 500 errors for six hours. Your RAG pipeline? Dead. Your frontend? Showing a pretty little "Connection Refused" message to anyone who tries to access it. And you? You thought everything was fine because it worked when you went to bed. You dumb beautiful bastard. You built a whole AI server — survived Docker, fought through quantization, locked down your network like a paranoid survivalist — and then you just... walked away. Blind. No gauges, no alarms, no fcking idea what was happening inside the machine. That ends today. Core Carnage (Rip Apart the Essentials) Monitoring is not optional. Let me say that again for the cheap seats: monitoring is not optional. You wouldn't drive a car with no dashboard — no speedometer, no fuel gauge, no check-engine light — and then act surprised when you're stranded on the highway at midnight. But that's exactly what you've been doing with your server. You've been flying blind and calling it "simpler." Here's the nightmare that monitoring solves: silent failure. Your containers are "running" — Docker says so. But "running" doesn't mean "working." A container can be technically alive and returning garbage. Your LLM endpoint can be up but timing out on every request. Your disk can be 97% full with everything still technically "functioning" until one log file pushes it over the edge and the entire filesystem goes read-only. I know what you're thinking: "I'll just check manually when something feels wrong." Cute. You know what "feels wrong"? When your Discord bot stops responding. When your partner complains the AI is slow. When you try to load a model and get CUDA out of memory on a Tuesday afternoon. By the time YOU notice, the damage is done. The real pros? They know about failures before anyone else does. Sometimes before the failure even fully happens. You need three things, and you need all three: 1. Uptime monitoring — "Is the thing alive?" Simple. Binary. Yes or no. This is your canary in the coal mine. 2. System monitoring — "Is the machine healthy?" CPU, RAM, disk, GPU, VRAM, network. The vitals. The stuff that tells you WHY something died, not just that it died. 3. Alerting — "Someone tell me about it." Because a dashboard you never look at is just art. You need to be paged. On your phone. At 2 AM. With enough context that you're not debugging blind. …
12. Autopilot: Compose, Update, Repeat
Picture this: It's 2 AM on a Saturday. Your buddy's at the bar, three drinks deep, telling everyone within earshot about his "sick AI server." Monday morning, a breaker trips. His server reboots. Nothing comes back up. No Ollama. No Open WebUI. No tunnel. He spends the next nine hours manually starting containers in the wrong order, googling error messages, and questioning every life decision that led him to this moment. That buddy? That was ME, twelve years ago. And if you skip this chapter, that buddy is gonna be YOU. You've made it. Eleven chapters of blood, sweat, and profanity. You picked hardware that doesn't suck. You installed a real OS. You learned Docker like your life depended on it. You deployed inference engines, wrangled models, bolted on a UI, wired up tunnels, locked it down, planned your storage, gave your AI a memory, and set up monitoring so you'd know when things went sideways. But here's the brutal truth: right now, your setup is a Jenga tower in a windstorm. It works because you're manually holding it together. One reboot, one bad update, one hardware swap, and you're back to spending a weekend in SSH hell, trying to remember which port goes where and which volume mount you forgot to document. Today, we fix that. Today, we turn your fragile art project into a self-healing war machine that survives reboots, updates itself, and can be rebuilt from scratch with a single command. This is the chapter where you stop being a guy who "runs some Docker stuff" and start being someone who runs a real infrastructure stack. Let's ride. Core Carnage (Rip Apart the Essentials) The One-Stack Religion Remember when you learned Docker back in "Containerize or Capitulate"? You were running individual docker run commands like some kind of caveman chipping flint. Then maybe you graduated to a basic compose file. Cute. But if you're like 90% of people who get this far, your setup has grown organically — and "organically" is just a pretty word for "messy as hell." You've probably got Ollama running one way, Open WebUI deployed another, your RAG vector database over here, your monitoring stack over there, each with its own little config file scattered across your filesystem like breadcrumbs in a fairy tale. Except the fairy tale ends with you losing three hours on a Tuesday night trying to figure out why Qdrant can't talk to Ollama after a restart. Here's the religion: One stack. One file. One command. Everything — and I mean EVERYTHING — goes into a single docker-compose.yml. Not two files. Not a "main" compose file and a "testing" one. ONE. If a service is part of your production …
Continue learning
- Docker for Beginners: Learn Containerization from ScratchDocker for Beginners: Learn Containerization from Scratch — a free beginner-level guide covering learn docker and containerization for beginners. Learn...
- Build A Local AI-Powered Semantic Search Engine For Your Personal Video ArchiveBuild A Local AI-Powered Semantic Search Engine For Your Personal Video Archive — a free advanced-level guide covering build a local ai-powered...
- Build A Local AI-Powered Video Search Engine With Python And CLIP EmbeddingsBuild A Local AI-Powered Video Search Engine With Python And CLIP Embeddings — a free advanced-level guide covering build a local ai-powered video...
- Build An AI-Powered Graph Database From Scratch For Advanced Fraud Detection And Network AnalysisBuild An AI-Powered Graph Database From Scratch For Advanced Fraud Detection And Network Analysis — a free advanced-level guide covering build an...