Pustakam Library

Free Programming learning guide

SQL for Data Analysis: A Beginner's Guide

SQL for Data Analysis: A Beginner's Guide — a free beginner-level guide covering learn sql for data analysis from scratch. Learn with clear...

102 min read12 chaptersbeginner

What you will learn

  1. Introduction to Relational Databases
  2. Retrieving Data with the SELECT Statement
  3. Filtering Data with the WHERE Clause
  4. Sorting and Organizing Result Sets
  5. Transforming Data with Built-in Functions
  6. Summarizing Data with Aggregations
  7. Combining Data from Multiple Tables with JOINs
  8. Handling Missing Values and Data Types
  9. Categorizing Data with Conditional Logic
  10. Writing Complex Queries with Subqueries
  11. Advanced Analysis with Window Functions
  12. Saving Queries with Views

1. Introduction to Relational Databases

The Hidden Engine Behind Everyday Decisions Imagine you are buying a coffee on your way to work. You tap your rewards app to pay. In the two seconds it takes for the barcode to scan, a system somewhere checks your identity, verifies your current account balance, applies a 10% member discount, logs the transaction, and updates your progress toward a free drink. This seamless everyday experience is powered by a relational database. Before data can be analyzed, it must be stored, organized, and made easily retrievable. As a data analyst, your job is rarely to build these massive systems from scratch. Instead, your job is to query them—to ask precise questions and extract the answers hidden within. To ask the right questions, you first need to understand how the data is structured. This chapter establishes the foundational concepts of how data is organized in a relational model, introduces the language used to interact with it, and guides you through setting up your own practice environment. What is a Relational Database? At its simplest, a database is an organized collection of information. A phone book is a database. A spreadsheet tracking your monthly expenses is a database. However, a relational database takes this concept further. Introduced by computer scientist Edgar F. Codd in 1970, the relational model organizes data into distinct, logical structures and defines strict rules for how different pieces of data relate to one another. Instead of dumping all information into one massive, unwieldy file, a relational database breaks data into manageable pieces. The Building Blocks: Tables, Columns, and Rows In a relational database, data is stored in tables. You can think of a table exactly like a single tab in a spreadsheet program like Microsoft Excel or Google Sheets. Every table in a relational database is dedicated to a specific entity—a single concept, person, place, or event. For example, an e-commerce company might have one table for customers, another for products, and a third for orders. Within each table, data is organized into rows and columns: Columns: These represent the attributes or properties of the data. If you have a customers table, your columns might be customerid, firstname, lastname, and email. Every row in that table will have these exact same columns. Rows: Also known as records, these represent a single, individual entry in the table. One row in the customers table contains the specific customerid, firstname, lastname, and email for exactly one person. Real-World Example: A Retail Company's Data Let’s look at how a retail company might structure a products table. | productid | productname | category | price | | :--- | :--- | :--- | :--- | | 101 | Stainless Steel Water Bottle | …

2. Retrieving Data with the SELECT Statement

Your First Conversation with the Database Imagine you are hired as a data analyst for an online bookstore. On your first day, your manager asks: "Can you pull up the list of all the books we currently have in inventory?" In a traditional spreadsheet program, you would simply grab your mouse, click on the tab labeled "Inventory," and scroll through the data. But you aren't working with a spreadsheet. You are working with a relational database. You cannot click on a table to open it; instead, you have to ask the database to hand the data to you. To ask a database for data, you use SQL, or Structured Query Language. A single instruction written in SQL is called a query. Think of a query as a highly specific, formal question you ask the database. If you ask the question correctly, the database will return a neat, formatted table as the answer. The foundational tool for asking any question in SQL is the SELECT statement. Whether you are looking up a single customer's name or analyzing millions of sales records, every data retrieval journey begins with SELECT. The Basic SELECT Statement At its simplest, the SELECT statement tells the database two things: what columns you want to see, and where to find them. Let's look at the absolute basic syntax for retrieving data from a single table: Let's break down this query word by word: SELECT: This keyword starts the query. It tells the database you want to retrieve data rather than insert, update, or delete it. (Asterisk): In SQL, the asterisk is a wildcard that means "all columns." By using the asterisk, you are telling the database to bring back every single column that exists in the table. FROM: This keyword specifies the location of the data. tablename: This is the name of the specific table you want to pull data from. ; (Semicolon): The semicolon marks the end of the SQL statement. While some database systems (like SQLite or PostgreSQL) will often run a query without a semicolon, including it is a best practice that ensures your query executes properly, especially when writing multiple queries in a row. Example 1: Viewing an Entire Table Let’s apply this to your new job at the online bookstore. The database has a table named books that stores the inventory. You want to see everything. You would write and execute the following query in your database browser tool (such as DB Browser for SQLite Online or DB Fiddle): When the database processes this query, it returns a result set—a temporary table displayed on your screen—containing every row and every column from the books table. You might see the book title, author, publication …

3. Filtering Data with the WHERE Clause

Imagine you are an inventory manager for a national electronics retailer. Your company’s database contains a single products table with over 50,000 rows—representing every keyboard, monitor, laptop, and cable across all your warehouses. If you write a basic SELECT statement to retrieve all rows and columns, your screen will be flooded with the entire catalog. Finding the specific laptop models that cost over $1,000 would mean manually scrolling through thousands of records. In the previous chapter, you learned how to use the SELECT statement to choose which columns to retrieve. However, data analysis rarely involves looking at an entire table at once. To make data analysis manageable and efficient, we need a way to retrieve only the records that matter to our specific question. This is where conditional logic comes in. By using the WHERE clause, you can instruct the database to return only the rows that meet specific criteria, effectively filtering out the noise. The WHERE Clause: Your Data Sieve In SQL, the WHERE clause acts as a filter. It evaluates every row in a table against a condition you define. If the condition is true for a row, that row is included in the result set. If the condition is false, the row is discarded. The basic syntax of a query with a WHERE clause looks like this: One of the most important rules to remember about SQL syntax is the order of operations. The WHERE clause must always come after the FROM clause. If you place it before, the database will return a syntax error because it doesn't know which table to filter until you specify it in the FROM clause. Exact Matches with Text and Numbers The simplest way to filter data is to ask the database for an exact match. You do this using the equals operator (=). Filtering Numbers When filtering by a number, you simply write the number as it is. Let’s say our retailer’s database has a products table, and we want to find every item that costs exactly $999. Filtering Text Filtering by text works the same way, but with one crucial difference: text values must be enclosed in single quotation marks ('). In SQL, single quotes indicate that the database should treat the characters as literal text, not as a command or a number. If we want to find all products in the "Audio" department, we write: Note: SQL is generally case-insensitive when it comes to keywords and table names, but the text inside your single quotes is often case-sensitive depending on the database system (like PostgreSQL). Always match the capitalization of the data exactly as it appears in the table. Comparison Operators Exact matches are helpful, but data analysis …

4. Sorting and Organizing Result Sets

The Natural Chaos of Databases Imagine walking into a library where thousands of books are scattered across the floor in the exact order they were returned. Finding a specific book, or even browsing books by your favorite author, would be nearly impossible. By default, relational databases operate like that messy library. When you retrieve data using a SELECT statement—and filter it down using a WHERE clause—the database returns the records in whatever order is fastest for it to access them at that moment. This order is dictated by the database's internal storage mechanics, not by what makes sense to you as a human. If you run the exact same query twice, there is no guarantee the results will appear in the same order. To make data useful for analysis, you need to impose order on it. This is where sorting and pagination come in. Sorting allows you to arrange your retrieved rows into a meaningful sequence, while pagination allows you to break up massive result sets into manageable, bite-sized pages. Sorting Data with ORDER BY The ORDER BY clause is the SQL command used to sort your result set. It is always placed at the very end of a query, after the SELECT and WHERE clauses. To use it, you simply specify the column you want to sort by. By default, SQL sorts data in ascending order—meaning from A to Z for text, from lowest to highest for numbers, and from earliest to latest for dates. Let’s look at an example. Imagine you are analyzing a table of company employees. You want to see a list of all employees who work in the Sales department, ordered alphabetically by their last name. Because we didn't specify a direction, SQL assumes ascending order. If an employee's last name is "Adams", they will appear at the top of the list. If someone is named "Zimmerman", they will be at the bottom. Controlling Direction: ASC and DESC While ascending order is the default, you will often want to reverse it. For instance, if you are looking at salaries, you usually want to see the highest earners first. To control the sort direction, you use the keywords ASC (ascending) or DESC (descending) immediately after the column name. If you want to find the Sales employees with the highest salaries, you would modify your query like this: Now, the employee making $150,000 will appear at the top, and the employee making $45,000 will be at the bottom. You can also mix and match directions when sorting multiple columns, but before we get to that, we need to look at how SQL handles a special type of data: empty values. Handling NULL Values During Sorting In the …

5. Transforming Data with Built-in Functions

The Power of Shaping Data on the Fly Imagine you are preparing a mailing list for a marketing campaign. You pull up your customers table and realize the data isn't quite ready for the printer. The first and last names are stored in separate columns, but the address labels need them combined into a single "Full Name" field. The product prices in your orders table have four decimal places, but you only need two for the currency format. Finally, you have a column of order timestamps, but you only want to see the year the order was placed to group them into annual cohorts. You could export the data into a spreadsheet, fix these formatting issues manually, and then proceed with your analysis. But doing this every time you need to look at the data is tedious and error-prone. This is where built-in functions come to the rescue. A function is a predefined, named block of code that takes an input, performs a specific operation on it, and returns a result. SQL includes a wide variety of built-in functions that allow you to transform and format data exactly as you need it, right at the exact moment you are retrieving it from the database. Scalar Functions vs. Aggregate Functions Before we dive into specific functions, it is crucial to understand that SQL groups its functions into two main categories: scalar functions and aggregate functions. A scalar function operates on a single row at a time. If you apply a scalar function to a text column, it will transform the text for each individual record independently. For example, if you use a function to uppercase a customer's name, that function runs once for every row in your result set. An aggregate function, on the other hand, takes data from multiple rows and crunches it down into a single summary value. Examples of aggregations include calculating the average price of all products or counting the total number of orders. This chapter focuses entirely on scalar functions. We will explore how they transform individual records by manipulating text, performing mathematical operations, and extracting pieces of dates. (We will save the deep dive into aggregate functions for the next chapter, where we tackle summarizing data). Transforming Text with String Functions Text data in a database is referred to as a string. Whether you are dealing with names, addresses, or product descriptions, strings often need to be cleaned up or reformatted. SQL provides a suite of string functions to handle this. Concatenating Strings Concatenation is the process of joining two or more strings together end-to-end. In SQL, the most universal way to concatenate strings is using the CONCAT() function. You pass the column names or …

6. Summarizing Data with Aggregations

Imagine you are handed a database containing 50,000 rows of daily sales transactions from an online electronics store. If you use the SELECT statement you learned in Chapter 2, you could retrieve every single row. But staring at 50,000 rows of data won't tell you whether your business is doing well. To make sense of that much data, you need to zoom out. You don't want to see individual transactions; you want to know the total revenue for the month, the average price of a laptop, or how many customers bought a specific mouse model. In SQL, the tool we use to zoom out is called an aggregation. Aggregation is the process of taking multiple rows of data and squashing them down into a single summary value. In this chapter, we will explore the built-in aggregate functions that make this possible, learn how to group those summaries by specific categories, and discover how to filter our data once it has been summarized. The Core Aggregate Functions An aggregate function performs a calculation on a set of values and returns one single value. SQL provides several core aggregate functions that cover the most common data analysis needs: counting, summing, averaging, and finding extremes. Let's use a simple sales table to demonstrate these functions. Imagine our table has three columns: transactionid, productcategory, and amount. | transactionid | productcategory | amount | | :--- | :--- | :--- | | 1 | Laptops | 1200.00 | | 2 | Accessories | 25.50 | | 3 | Laptops | 950.00 | | 4 | Accessories | 45.00 | | 5 | Laptops | 1300.00 | COUNT() The COUNT() function does exactly what it sounds like: it counts rows. It is incredibly useful for answering questions like "How many orders did we get today?" or "How many products are in our catalog?" You can count every row in a table using an asterisk: This query returns 5, because there are five rows in our table. You can also count the number of non-empty (non-NULL) values in a specific column. We will cover missing values (NULLs) extensively in Chapter 8, but for now, just know that COUNT(columnname) only counts rows that actually have data in that column. SUM() When you need a total, you use SUM(). This function adds up all the numeric values in a specific column. It only works on numbers—if you try to sum a column of text, the database will throw an error. To find the total revenue from all our sales: This adds 1200 + 25.50 + 950 + 45 + 1300, returning a single value: 3520.50. AVG() The AVG() function calculates the average (the mean) of a numeric column. It …

7. Combining Data from Multiple Tables with JOINs

Why Data Lives in Separate Places Imagine running an online store. Every time a customer places an order, you could theoretically record everything—the customer’s name, their address, the product they bought, the product’s price, and the order date—into one massive spreadsheet. But within a few months, that spreadsheet would become a nightmare. If a customer moved and updated their address, you would have to find and update every single row where they ever placed an order. If you mistyped a product name, you’d have to fix it hundreds of times. This is why relational databases avoid storing all data in one giant table. Instead, they spread data across multiple smaller, specialized tables. You might have a customers table, a products table, and an orders table. This separation prevents redundancy and keeps data accurate. However, when it comes time to analyze that data—say, figuring out which customers bought which products—you need a way to stitch those separate tables back together. In SQL, the tool for stitching tables together is the JOIN clause. Before we can combine tables, we need to understand the "thread" that connects them: keys. The Glue That Holds Tables Together: Keys In a relational database, tables connect to one another using common columns. These columns are known as keys. There are two types of keys you need to know about to start joining tables: primary keys and foreign keys. Primary Keys A primary key is a column (or set of columns) in a table that uniquely identifies every single row in that table. Think of it as a social security number or a fingerprint for a row. A primary key must contain unique values, and it cannot be empty (NULL). For example, in a customers table, you might have a column called customerid. Because every customer gets their own unique ID, customerid is the perfect primary key. Foreign Keys A foreign key is a column in one table that references the primary key of another table. This is the actual link between the two tables. Let’s look at an orders table. Every order needs to be associated with the customer who placed it. Instead of writing the customer's full name and address into the orders table, you simply include a customerid column. In the orders table, customerid is a foreign key—it points back to the primary key in the customers table. By matching the customerid in the orders table to the customerid in the customers table, SQL knows exactly which customer placed which order. The INNER JOIN: Finding the Match The most common way to combine tables is the INNER JOIN. An INNER JOIN returns only the rows where there is a match in both tables based on …

8. Handling Missing Values and Data Types

The Ghosts in Your Data: Understanding NULL Imagine you are analyzing a customers table for an e-commerce company. You write a query to calculate the average order value, but the number comes out much lower than expected. You try to find all customers who haven't provided a phone number, but your query returns zero rows—even though you know dozens of customers leave that field blank. The culprit in both scenarios is likely NULL, SQL’s way of representing missing, unknown, or inapplicable data. In previous chapters, we used aggregations to summarize data and built-in functions to transform text and dates. Now, we need to learn how to handle the gaps in our data. Real-world data is rarely perfect. Customers skip optional fields, sensors fail to record readings, and systems experience glitches. To do accurate data analysis, you must learn how to identify, handle, and work around these missing values. What Exactly is a NULL? In SQL, a NULL is not the same as zero, and it is not the same as an empty text string. - Zero is a specific numerical value. If a bank account balance is 0, the database knows exactly how much money is in the account. - An empty string ('') is a text value that contains no characters. It takes up space in memory and is a deliberate entry of "nothing" in a text field. - NULL means the absence of a value. It is a state of unknown. If a customer's age column is NULL, it doesn't mean the customer is zero years old; it means we simply do not know their age. How NULL Behaves in Queries Because NULL represents an unknown state, it behaves unpredictably if you treat it like a normal value. This behavior trips up almost every beginner SQL user. NULL in the WHERE Clause In Chapter 3, we learned how to filter data using the WHERE clause with operators like =, <, and . However, you cannot use the equals sign to find NULL values. If you write WHERE phonenumber = NULL, the database does not look for missing phone numbers. Instead, it asks, "Does an unknown value equal an unknown value?" The answer to that is also unknown, so the database returns zero rows. To properly filter for NULLs, you must use the special operators IS NULL or IS NOT NULL. NULL in Aggregations In Chapter 6, we used functions like SUM(), AVG(), and COUNT() to summarize data. When these functions encounter NULLs, they simply ignore them. If you have five orders with values of $10, $20, NULL, $30, and $40: - SUM(ordertotal) will add 10 + 20 + 30 + 40 = $100. - AVG(ordertotal) will divide the …

9. Categorizing Data with Conditional Logic

The "If-Then" Logic Your Data Is Missing Imagine you are an analyst at an online retail company. You have a customers table with a column called totalpurchases. You want to send a promotional email, but the discount code depends on how much they’ve spent: customers who spent over $500 get a "Gold" code, those who spent between $100 and $500 get a "Silver" code, and everyone else gets a "Bronze" code. If you were using a spreadsheet, you would likely write a formula using IF or IFS. But how do you do this in SQL? Up to this point in our SQL journey, we have learned how to filter rows using the WHERE clause, transform text and dates with built-in functions, and summarize numerical data using aggregations. But what if we want to create entirely new categories of data on the fly, or change how data is displayed based on specific rules? To do this, SQL uses conditional logic—a way to execute different actions based on whether a specific condition is true or false. The primary tool for implementing this if-then-else logic in SQL is the CASE statement. The Basic CASE Statement A CASE statement evaluates a list of conditions and returns a specific value as soon as it finds the first condition that is true. Think of it as a series of "If... Then..." questions you ask the database. Here is the basic syntax: Let’s break down the jargon: WHEN: This sets up your condition (e.g., "If the total purchases are greater than 500"). THEN: This defines what happens when the WHEN condition is true (e.g., "Then label them 'Gold'"). ELSE: This catches everything that didn't meet any of the WHEN conditions. It is optional, but highly recommended. END: Every CASE statement must literally end with the word END so SQL knows your logic is complete. Real-World Example: Categorizing Customer Spending Let’s apply this to our retail scenario. We want to categorize our customers based on their totalpurchases in the customers table. How the database reads this: SQL looks at the first row. It checks the first WHEN condition. Is totalpurchases greater than 500? If yes, it immediately outputs 'Gold' and moves to the next row. If no, it moves to the second WHEN condition. Is it greater than or equal to 100? If yes, it outputs 'Silver'. If neither of those is true, it falls back to the ELSE clause and outputs 'Bronze'. We used the AS keyword (which we learned in Sorting and Organizing Result Sets) to give our newly created column a readable name: customertier. Note on ordering: The sequence of your WHEN clauses matters. SQL evaluates them top-to-bottom. If you put the = 100 condition …

10. Writing Complex Queries with Subqueries

What is a Subquery? Imagine you are an analyst at an online electronics store. Your manager asks you to pull a list of all products that are priced higher than the store's average product price. If you rely only on the tools we covered in earlier chapters, you might feel stuck. In Summarizing Data with Aggregations, you learned how to calculate an average using the AVG() function. In Filtering Data with the WHERE Clause, you learned how to filter numbers using operators like . However, the WHERE clause cannot directly contain aggregate functions. You cannot simply write WHERE price AVG(price). You could run one query to find the average price (say, $250), look at that number, and then manually type it into a second query: WHERE price 250. But what if the database updates every few minutes? That manual average would quickly become outdated. This is where subqueries come to the rescue. A subquery (also called an inner query or nested query) is a query embedded inside another SQL query. It allows you to perform multi-step operations in a single trip to the database. The subquery executes first, gathers the necessary information, and hands that result off to the outer query (the main query) to finish the job. Subqueries are incredibly versatile. You can place them in three different parts of a SQL statement: 1. The WHERE clause (for multi-step filtering) 2. The SELECT clause (for calculating new columns) 3. The FROM clause (for querying the results of another query) Subqueries in the WHERE Clause Using a subquery in the WHERE clause is the most common way to filter data based on the results of another query. Let’s revisit our opening scenario to see how it works. Comparing Against Aggregated Results We need to find products priced higher than the average product price. Here is how we write this as a nested query: Let’s break down how the database reads this: 1. The database sees the parentheses inside the WHERE clause and recognizes a subquery. 2. It executes the inner query first: SELECT AVG(price) FROM products;. This calculates the overall average price of all products (let's assume it returns 250). 3. The database conceptually replaces the subquery with that value. The outer query effectively becomes: SELECT productname, price FROM products WHERE price 250;. 4. It executes the outer query and returns the results. By wrapping the AVG(price) calculation inside a subquery, we solved the rule that prevents aggregate functions from being used directly in a WHERE clause. Filtering Using the IN Operator Sometimes, you don't want to compare against a single calculated value like an average. Instead, you want to filter against a list of values. Suppose you want to …

11. Advanced Analysis with Window Functions

The Limits of Standard Aggregation Imagine you are an analyst at an online retail company. Your manager hands you a dataset of recent customer purchases and asks a seemingly simple question: "Show me every purchase made in the last month, but next to each purchase, include the total amount that specific customer has spent overall." If you are relying solely on the standard aggregation tools we covered in Summarizing Data with Aggregations, you will hit a wall. If you use the SUM() function with a GROUP BY clause, you can easily find the total spent per customer. However, doing so collapses the individual rows. Your output will show one row per customer with their total, completely erasing the individual purchase records your manager requested. To solve this, you might turn to Writing Complex Queries with Subqueries. You could write a query that joins the purchases table to a subquery that calculates the customer totals. While this works, it is verbose and can be computationally slow on large datasets. This is where window functions come to the rescue. A window function performs a calculation across a set of table rows that are somehow related to the current row. The magic of window functions is that they perform these calculations without collapsing the output. You get the aggregated calculation, but you keep all the individual, detailed rows intact. Understanding the OVER() Clause Every window function relies on one core component: the OVER() clause. This clause tells the SQL database, "Instead of grouping all these rows together into one result, calculate this function over a specific 'window' of related rows and attach the result to my current row." If you write OVER() with empty parentheses, you are telling SQL to look at the entire result set as the window. Let's look at an example using a simplified sales table. | saleid | employeeid | saledate | amount | | :--- | :--- | :--- | :--- | | 1 | 101 | 2023-10-01 | 150.00 | | 2 | 102 | 2023-10-01 | 200.00 | | 3 | 101 | 2023-10-02 | 50.00 | If we want to see every individual sale alongside the total amount of all sales combined, we write: Output: | saleid | employeeid | saledate | amount | grandtotal | | :--- | :--- | :--- | :--- | :--- | | 1 | 101 | 2023-10-01 | 150.00 | 400.00 | | 2 | 102 | 2023-10-01 | 200.00 | 400.00 | | 3 | 101 | 2023-10-02 | 50.00 | 400.00 | Notice what happened. The SUM(amount) function did its usual job of adding up the numbers. But instead of collapsing three rows into one, the OVER() …

12. Saving Queries with Views

The Problem with Repetitive Complex Queries Imagine you are a data analyst at an e-commerce company. Every Monday, the marketing team asks you the same question: "What were our top 10 best-selling product categories last week, and who were the top spending customers in those categories?" To answer this, you have to write a massive SQL query. You need to use JOINs to connect the sales, products, and customers tables. You need to use Aggregations like SUM() to calculate total sales. You need to use Window Functions like RANK() to find the top 10 categories. You might even need a Subquery to filter the final results. Writing this query took you an hour the first time. Now, you have to open your old SQL file, copy the 30-line query, paste it into your database tool, and run it. If you make a tiny typo while copying it, the query breaks. Worse, what happens if the marketing team wants to run this report themselves? They don't know SQL, and you don't want to give them access to raw, underlying tables where they might accidentally delete data. This is where views come to the rescue. What is a View? In SQL, a view is a virtual table based on the result set of a saved query. Let’s break down that definition: - Virtual table: To anyone querying it, a view looks and acts exactly like a regular table. It has columns and rows. However, unlike a real table, a view does not store the data itself on the hard drive (with a few exceptions we will discuss later). - Saved query: A view is essentially a SQL SELECT statement that has been given a name and saved permanently in the database. Think of a view as a saved bookmark for a complex query. Instead of typing out a 30-line query every Monday, you save it once as a view named weeklymarketingreport. From then on, you can just write SELECT FROM weeklymarketingreport;. Why Are Views Useful for Data Analysts? Views are not just a convenience; they are a core part of how professional databases are managed. For a data analyst, they offer four major benefits: 1. Simplicity and Reusibility: You can wrap complex logic—like multi-table joins and window functions—into a single, clean view. You write the complex logic once, save it, and never have to write it again. 2. Security: Databases often contain sensitive information (like a customer's password or social security number). A view allows you to expose only the specific columns a user needs to see, hiding the underlying sensitive data. 3. Consistency: If five different analysts on your team need to calculate "Net Revenue," they might calculate it five slightly …

Continue learning