Free Programming learning guide
Python for Data Science: A Beginner's Roadmap
Python for Data Science: A Beginner's Roadmap — a free beginner-level guide covering how to learn python for data science. Learn with clear...
What you will learn
- Python Basics and Environment Setup
- Core Data Structures
- Control Flow and Functions
- Jupyter Notebooks for Data Science
- Numerical Computing with NumPy
- Data Manipulation with Pandas (Part 1)
- Data Manipulation with Pandas (Part 2)
- Data Visualization with Matplotlib and Seaborn
- Exploratory Data Analysis and Basic Statistics
- Introduction to Machine Learning with Scikit-Learn
1. Python Basics and Environment Setup
Why Python for Data Science? Imagine you are handed a massive spreadsheet containing five years of daily sales data from a chain of 50 retail stores. The file is too large to open in standard spreadsheet software, and your manager wants a report by tomorrow highlighting seasonal trends, average transaction values, and the worst-performing locations. Manually scrolling through rows is impossible. You need a way to instruct a computer to read, calculate, and summarize that data for you. This is where Python comes in. Python is a general-purpose programming language—meaning it isn't specialized for just one task. You can use it to build websites, program video games, and, crucially for us, analyze massive datasets. It has become the undisputed language of choice for data science because of its gentle learning curve and an enormous ecosystem of free, pre-written tools (called "libraries") designed specifically for data manipulation, visualization, and machine learning. Before we can analyze data, we need to set up our workspace and learn the basic grammar of the language. Think of this chapter as setting up your kitchen and learning how to hold a knife before you attempt to cook a five-course meal. Setting Up Your Environment To write and run Python code, you need two things installed on your computer: the Python interpreter and a code editor. Installing the Python Interpreter A computer cannot inherently understand Python. It needs a translator to convert the Python text you write into the 1s and 0s the computer's processor understands. This translator is the Python interpreter. 1. Go to the official Python website: python.org. 2. Navigate to the "Downloads" section. The site should automatically detect your operating system (Windows, macOS, or Linux) and recommend the latest version of Python. 3. Download the installer and run it. Important Windows Step: During the installation process on Windows, you will see a checkbox at the bottom of the first setup screen that says "Add Python to PATH". Make sure this box is checked. PATH is a system variable that tells your computer where to find executable programs. If you skip this, your computer won't know where Python is located when you try to run it later. Installing a Code Editor You can technically write Python code in a basic text editor like Notepad or TextEdit, but this is like writing a novel on a napkin. A code editor is a specialized text editor designed for programming. It provides features like syntax highlighting (coloring your code to make it readable), auto-indentation, and tools to run your code directly. For this book, we will use Visual Studio Code (VS Code). It is free, powerful, and widely used by professional developers and data scientists. 1. Go to …
2. Core Data Structures
Imagine you are preparing to analyze a dataset containing information about 10,000 different houses. If you had to create a new variable for every single piece of data—price1, price2, price3, bedrooms1, bedrooms2—your script would quickly become an unmanageable nightmare. In Python Basics and Environment Setup, you learned how to store a single piece of data—a number like an integer or float, or a piece of text like a string—inside a variable. But in data science, you rarely work with single, isolated pieces of information. You work with collections of data. Python provides built-in data structures—specialized containers designed to organize, store, and manipulate multiple pieces of data at once. In this chapter, we will explore the three most fundamental built-in structures: lists, dictionaries, and tuples. Lists: Ordering Your Data A list is a data structure that stores an ordered collection of items. You can think of it as a single variable that holds a sequence of other variables. To create a list, you use square brackets [] and separate each item with a comma. Because we already know how to use string literals, let's create a list of city names as strings: You can put any data type into a list, and you can even mix them. For example, you might have a list representing a single row of housing data: This list contains a string (the neighborhood type), an integer (bedrooms), a float (bathrooms), and another integer (price). Accessing Elements via Indexing Once data is in a list, how do you get it back out? Python uses a concept called indexing. Every item in a list is assigned a position number, called an index, starting at 0. This zero-based counting is a fundamental convention in programming. The first item is at index 0, the second is at index 1, and so on. You access an item by writing the list name followed by the index in square brackets. Python also allows negative indexing, which lets you count backward from the end of the list. The index -1 always refers to the last item, -2 refers to the second-to-last, and so on. Modifying Lists Unlike some data structures we will look at later, lists are mutable. This means you can change their contents after you create them. To change an existing item, you assign a new value to a specific index using the assignment operator =. You can also add entirely new items to a list. To add an item to the very end, you use the .append() function. Slicing Lists Sometimes you don't want just one item; you want a subset of the list. Python lets you extract a portion of a list using a technique called slicing. Slicing uses …
3. Control Flow and Functions
Making Decisions in Your Code Imagine you are writing a script to analyze customer data. You want to categorize customers based on their age, but a customer’s age isn't a fixed value—it changes depending on who the script is currently looking at. Up to this point, your code has run strictly top-to-bottom, executing every line exactly as written. To build intelligent data workflows, your code needs the ability to make decisions and adapt to different situations. This ability is called control flow. It refers to the order in which individual statements, instructions, or function calls are executed in a script. By using control flow, you stop being a typist of linear instructions and start acting like an architect, designing the logical pathways your data will travel through. Conditional Statements: if, elif, and else The most fundamental tool for directing logic is the conditional statement. A conditional statement tells Python to execute a specific block of code only if a certain condition is met. Python evaluates conditions using comparison operators, which compare two values and result in either True or False. You may recognize these from basic math, with a few programming-specific tweaks: - == (Equal to): Notice the double equals sign. Remember from Chapter 1 that a single = is the assignment operator, used to assign values to variables. == is used to check if two things are equal. - != (Not equal to) - (Greater than) and < (Less than) - = (Greater than or equal to) and <= (Less than or equal to) Let’s look at how these come together using if, elif, and else. 1. The if statement The if statement initiates a conditional block. If the condition evaluates to True, the indented code block runs. If it evaluates to False, Python skips that block entirely. Notice the colon : at the end of the if statement. This is required syntax in Python. Also, notice that print("Customer is an adult.") is indented. In Python, indentation is not just for readability; it defines the block of code that belongs to the if statement. If you forget the indentation or the colon, the Python interpreter will stop and throw an error message. 2. The else statement Sometimes you want a fallback plan—a block of code that runs only if the if condition is False. This is where else comes in. 3. The elif statement Often, you have more than two possible outcomes. The elif statement (short for "else if") allows you to check multiple conditions in sequence. Python checks them from top to bottom. As soon as it finds one that is True, it runs that block and skips the rest. You can chain as many elif statements together …
4. Jupyter Notebooks for Data Science
Why a Different Environment for Data Science? Imagine you are handed a spreadsheet containing 50,000 rows of customer purchase history. You want to calculate the average order value, group the data by region, and look for seasonal trends. If you were to write a traditional Python script in Visual Studio Code (VS Code) to do this, your workflow would look something like this: write the code to load the data, run the script, print the results to the terminal, realize you made a mistake in your calculation, edit the code, run the entire script again, and print the results again. For small programs, this works perfectly well. But for data science, where exploration is highly interactive and trial-and-error is the norm, this cycle is painfully slow. Data scientists need to load a massive dataset into memory once, look at it, test a quick calculation, plot a graph, and then write a function—all without losing the data they just spent ten seconds loading. Enter Jupyter Notebook. A Jupyter Notebook is an interactive, web-based computational environment that allows you to combine executable Python code, the output of that code (like graphs or tables), and formatted text into a single document. Instead of running an entire script from top to bottom, you write code in small, manageable chunks called "cells." You can run one cell, look at the result, run the next cell, and keep going. The variables you create in one cell stay in the computer's memory, ready to be used in the next cell. The name "Jupyter" is a nod to three core programming languages it supports: Julia, Python, and R. While it supports all three, it is overwhelmingly used by the Python community. Installing and Launching Jupyter Notebook Because Jupyter Notebook runs in your web browser, you might assume it is a website you visit. In reality, the Jupyter server runs locally on your computer. It uses your browser only as an interface to display the files and code. The Installation Process If you followed the setup steps from Python Basics and Environment Setup, you already have Python installed on your computer. To get Jupyter Notebook, the easiest and most standard method is to use Python’s package manager, pip. Open your terminal (Command Prompt or PowerShell on Windows, Terminal on macOS/Linux) and type the following command: Press Enter. You will see a series of progress bars as Python downloads and installs Jupyter and its required background files. (Note: Another common installation method is through the Anaconda Distribution, a massive bundle of data science tools. If you installed Anaconda previously, Jupyter is already installed, and you can launch it from the Anaconda Navigator application. For this chapter, we will assume the …
5. Numerical Computing with NumPy
Why We Need a New Tool for Numbers Imagine you are analyzing the daily temperature readings from 10,000 weather stations across the country. You have a list of 10,000 numbers, and you need to convert them all from Fahrenheit to Celsius. Based on what we covered in Core Data Structures, you might write a for loop to iterate through the list, apply the conversion formula to each number, and append the result to a new list. This approach works perfectly, but it is slow. Python is a general-purpose programming language, which means it is designed to handle everything from web servers to text processing. Because of this flexibility, Python stores every number in a standard list as a full-fledged object with overhead (like type information and reference counting). When you loop through 10,000 numbers, Python has to look up the type and process the overhead of every single item, one at a time. NumPy (short for Numerical Python) solves this problem. It is a specialized library designed specifically for performing fast mathematical operations on large arrays of numbers. It bypasses Python's usual overhead by using highly optimized C code under the hood. Instead of processing 10,000 numbers one by one, NumPy can perform operations on all of them simultaneously in a fraction of a second. To use NumPy, we first need to import it. By strong convention, the data science community imports NumPy using the alias np: Whenever you see np. in code, you will know we are using a tool from the NumPy library. Creating NumPy Arrays The core data structure in NumPy is the array (specifically, an ndarray, which stands for N-dimensional array). While a standard Python list can hold a mix of integers, floats, and strings, a NumPy array is homogeneous—it can only contain data of the exact same type (usually numbers). This restriction is exactly what makes NumPy so fast. From Python Lists to NumPy Arrays The most common way to create a NumPy array is by converting a standard Python list. We do this using the np.array() function. Output: Notice how the output looks slightly different than a Python list—there are no commas separating the values. This is a quick visual cue that you are working with a NumPy array rather than a standard list. You can also create arrays from scratch without starting with a list. NumPy provides built-in functions to generate common patterns of numbers: - np.zeros(n): Creates an array of n zeros. - np.ones(n): Creates an array of n ones. - np.arange(start, stop, step): Works like Python's range() function, but returns a NumPy array. Performing Element-wise Mathematical Operations The true power of NumPy lies in a concept called vectorization. Vectorization allows you …
6. Data Manipulation with Pandas (Part 1)
Why Pandas? Imagine you just downloaded a dataset containing 50,000 rows of housing prices. You need to find the average price of homes with three bedrooms, figure out which rows are missing square footage data, and isolate just the columns for price and zip code. If you were working with standard Python lists and dictionaries—the Core Data Structures we covered earlier—this would be a grueling process. You would need to write for loops, manually track indices, and build entirely new lists to hold your filtered data. Data scientists deal with this exact scenario daily. To handle tabular data (data that fits neatly into rows and columns, like a spreadsheet) efficiently, the Python community built a library called Pandas. Because Pandas is built on top of NumPy, it inherits the blazing-fast numerical computing power we explored in the previous chapter, but wraps it in a structure that is highly intuitive for data analysis. Pandas and DataFrames: The Building Blocks Before we write any code, we need to define two terms that will appear constantly in your data science journey: Pandas: An open-source Python library specifically designed for data manipulation and analysis. DataFrame: The core data structure in Pandas. You can think of a DataFrame as a spreadsheet or a database table living inside your Python environment. It has rows, columns, and built-in methods to quickly slice, filter, and summarize the data. To use Pandas, we first need to import it. By universal convention, data scientists import Pandas using the alias pd. Open a new Jupyter Notebook and run the following in your first cell: By using the alias pd, we don't have to type out the full word pandas every time we want to use one of its functions. Loading Data into a DataFrame Data comes in many formats, but the most common format for beginner data science projects is the CSV (Comma-Separated Values) file. A CSV is a plain text file where each line represents a row of data, and a comma separates each column. Pandas provides a highly optimized function, pd.readcsv(), to load these files into a DataFrame. Real-World Example 1: Loading E-commerce Data Let’s imagine you are working for an online retailer. You have a CSV file named salesdata.csv containing customer transactions. Here is how you load it: Notice how we use the assignment operator (=) to assign the resulting DataFrame to the variable salesdf. What if the file isn't in the same folder as your script? If your data is stored elsewhere on your computer, you need to provide the full file path. Important Windows Step: When typing file paths as a string literal in Python, backslashes (\) can cause errors because Python interprets them as escape …
7. Data Manipulation with Pandas (Part 2)
The Messy Reality of Data Imagine you are handed a spreadsheet containing 50,000 customer orders from an online store. Your manager asks a seemingly simple question: "What is the average order value for customers in California, excluding the test transactions?" You open the file and immediately hit a wall. Some rows are missing the customer's state. A few order values show up as negative numbers due to refund glitches. The data is sorted by product ID, making it impossible to look at California customers together. And scattered throughout the dataset are entirely blank rows where the website crashed during checkout. Raw data is almost never ready for analysis the moment it arrives. It is messy, incomplete, and unorganized. In Data Manipulation with Pandas (Part 1), we learned how to load data into a DataFrame and select specific rows and columns. Now, we will take the next step: cleaning and transforming that data so it actually answers our questions. To do this, we will rely heavily on the pandas library. If you are following along in a Jupyter Notebooks for Data Science environment, make sure to import pandas first: Throughout this chapter, we will use a sample dataset to practice our skills. Let's create a small DataFrame representing orders from that fictional online store: Handling Missing Data Missing data is a fact of life in data science. When pandas loads a dataset, it looks for empty cells in your spreadsheet or null values in a database. When it finds them, it fills them with a special object called NaN, which stands for "Not a Number". You can see an example of this in our df above: David's State is missing, and Eve's OrderValue is missing. Before we can analyze this data, we have to decide what to do with these NaN values. Generally, you have two choices: drop them or fill them. Dropping Missing Values If you have a massive dataset and the missing values represent a tiny fraction of your total data, the easiest approach is to simply remove those rows. Pandas provides the dropna() function for this. If you run this, you'll notice that the rows for David (missing State) and Eve (missing OrderValue) are completely removed. Sometimes, you only want to drop a row if a specific column is missing data. You can do this using the subset parameter: Filling Missing Values Dropping data isn't always the best choice. If you drop every row with a missing value, you might accidentally throw away half your dataset! Instead, you can fill those empty cells with a specific value using the fillna() function. This is highly common when dealing with survey data or numerical features where a blank actually implies …
8. Data Visualization with Matplotlib and Seaborn
Why Visualize Data? Imagine staring at a spreadsheet containing 10,000 rows of daily temperature readings from 50 different cities over the last decade. Even if you are a master of Pandas, using .head() or .describe() will only give you a tiny, numerical snapshot of that data. The human brain, however, is exceptionally good at processing visual information. If you plot those temperatures on a graph, a sudden spike, a seasonal dip, or a long-term upward trend becomes instantly obvious. Data visualization is the process of translating raw numbers into graphical representations. In data science, it serves two primary purposes: exploration (helping you find patterns, trends, and errors in your data) and communication (sharing your findings with others). In this chapter, we will use two of the most popular Python libraries for this task: Matplotlib and Seaborn. Matplotlib is the foundational plotting library in Python—it gives you total control over every element of a plot. Seaborn is a library built on top of Matplotlib—it provides a higher-level interface specifically designed for statistical data visualization, making complex plots easier to create and more aesthetically pleasing by default. Getting Started with Matplotlib Matplotlib is a third-party library, meaning it doesn't come built into standard Python. If you set up your environment using Anaconda (as discussed in Python Basics and Environment Setup), Matplotlib is already installed. If you are using a standalone Python installation, you can install it via your terminal using pip install matplotlib. We will also use Pandas to prepare our data, so ensure you have that ready as well. To start creating visualizations, we need to import the library. The standard convention in the data science community is to import the pyplot module from Matplotlib and alias it as plt: Creating Basic Line Charts A line chart connects a series of data points with a continuous line. It is the best choice for showing how a variable changes over time. Let’s look at a real-world scenario: tracking the monthly revenue of a small coffee shop over its first year of business. We will store our months as a list of strings and our revenue as a list of integers (concepts covered in Core Data Structures). When you run this code in a Jupyter Notebook (or a standard Python script), the plt.plot() function creates the line chart, and plt.show() renders it. Without plt.show(), the plot may not display properly in some environments, though Jupyter Notebooks often display it automatically. Creating Bar Charts While line charts show continuous change, bar charts are used to compare discrete categories. A bar chart uses rectangular bars where the length of each bar is proportional to the value it represents. Let’s visualize the coffee shop’s top three …
9. Exploratory Data Analysis and Basic Statistics
The Detective Work of Data Science Imagine you are a real estate agent handed a spreadsheet containing the sale prices of 10,000 houses sold in your city last year. If someone asks you, "What is the typical house price?" or "Do bigger houses actually cost more?" you cannot possibly find the answer by staring at 10,000 rows. You need a way to summarize, condense, and explore the data to uncover its underlying story. This process is known as Exploratory Data Analysis (EDA). EDA is the detective work of data science. Before you can make predictions or build complex machine learning models, you must understand what your data looks like, where its outliers are, and how different pieces of information relate to one another. In previous chapters, you learned how to clean and manipulate data using Pandas, and how to plot it using Matplotlib and Seaborn. Now, we will combine those tools with fundamental statistical concepts. By calculating mathematical summaries, we can replace thousands of rows of raw numbers with a few key metrics that tell us exactly what is happening. The Center of Attention: Measures of Central Tendency The first question we usually ask of a dataset is: "What is the typical value?" In statistics, this is called a measure of central tendency. It tells us where the middle or center of our data lies. The two most common measures are the mean and the median. The Mean (Average) The mean is what most people think of when they hear the word "average." You calculate it by adding up all the values in a dataset and dividing by the total number of values. If we have a small sample of house prices: $150,000, $200,000, $250,000, and $300,000, the mean is $225,000. In Pandas, calculating the mean is straightforward using the .mean() method on a specific column. If you have a DataFrame named df with a column called Price, you simply write df['Price'].mean(). The Median The median is the middle value when your data is sorted from lowest to highest. If there is an even number of data points, the median is the average of the two middle numbers. Why use the median instead of the mean? Because the mean is highly sensitive to outliers—values that are unusually high or low compared to the rest of the data. Real-World Example: The Billionaire Next Door Imagine a neighborhood with five houses. Four of them are valued at $200,000, and the fifth is a mega-mansion owned by a billionaire, valued at $10,000,000. The mean house price would be $2,160,000. The median house price would be $200,000. If you were writing an article about the housing market in this neighborhood, which number better represents …
10. Introduction to Machine Learning with Scikit-Learn
Imagine you are trying to sell your house. You know the square footage, the number of bedrooms, the age of the property, and the zip code. How do you determine the right asking price? You could look at recent sales in your neighborhood, mentally weigh how each feature affects the value, and arrive at an educated guess. This is exactly what machine learning does, but at a scale and speed impossible for humans to match. In Exploratory Data Analysis and Basic Statistics, you learned how to inspect data, find patterns, and summarize datasets using Python and Pandas. Now, we are going to take the next step: teaching the computer to learn from that historical data so it can make predictions about new data. This chapter introduces Scikit-Learn, Python’s premier library for traditional machine learning. We will build our first predictive models using pre-built algorithms, starting from the ground up. What is Machine Learning? At its core, machine learning is a branch of artificial intelligence where computers learn from data without being explicitly programmed. If you wanted to write a traditional Python script to detect spam emails, you would have to write endless if statements: if the email contains "free money", if the sender is unknown, if the subject line is in all caps... mark it as spam. A machine learning approach is different. You feed the computer thousands of emails labeled "spam" and thousands labeled "not spam." The algorithm figures out the patterns on its own. Going forward, when it sees a new email, it uses those learned patterns to predict which category it belongs to. Supervised vs. Unsupervised Learning Machine learning algorithms generally fall into two main categories: supervised and unsupervised learning. Supervised Learning In supervised learning, the algorithm learns from labeled data. This means the dataset you provide already contains the answers. For example, imagine a dataset of past house sales. The "input" features are square footage, bedrooms, and age. The "output" (the label or target) is the final sale price. The algorithm's job is to figure out the mathematical relationship between the inputs and the output. Once it figures out that relationship, you can give it the inputs for a new house, and it will predict the output (the price). Supervised learning is divided into two main tasks: Classification: Predicting a category. (e.g., Is this email spam or not spam? Is this tumor malignant or benign?) Regression: Predicting a continuous number. (e.g., What will the temperature be tomorrow? What is the estimated value of this car?) Unsupervised Learning In unsupervised learning, the data does not have labels. The algorithm is given inputs but no outputs. Its job is to discover hidden structures or groupings within the data …
Continue learning
- Intermediate Python Projects for Portfolio BuildingIntermediate Python Projects for Portfolio Building — a free intermediate-level guide covering intermediate python projects for portfolio building....
- Python Programming for Beginners: Zero to Real ProjectsPython Programming for Beginners: Zero to Real Projects — a free beginner-level guide covering learn python programming from zero to real projects....
- Python Programming for Beginners: From Zero to Real ProjectsPython Programming for Beginners: From Zero to Real Projects — a free beginner-level guide covering learn python programming from zero to real...
- Python for Beginners: From Zero to Real-World ProjectsPython for Beginners: From Zero to Real-World Projects — a free beginner-level guide covering learn python programming from zero to real projects....