The head of growth’s question needs columns from four tables. Revenue lives in order_items. Whether an order was cancelled, and when it was placed, lives in orders. The product’s category lives in products, and the customer’s country in customers. In a spreadsheet you would write lookup formulas; in pandas you merge. Merges are where most silent errors in analysis happen, so this lesson spends as much time on checking a merge as on writing one.
import pandas as pd\n\nitems = pd.read_csv(\"order_items.csv\")\norders = pd.read_csv(\"orders.csv\", parse_dates=[\"order_date\"])\nitems[\"revenue\"] = items[\"quantity\"] * items[\"unit_price\"] * (1 - items[\"discount\"])\n\nlines = items.merge(orders, on=\"order_id\", how=\"left\", validate=\"many_to_one\")\nprint(len(items), \"->\", len(lines))\nprint(lines[[\"order_item_id\", \"order_id\", \"revenue\", \"order_date\", \"status\"]].head(3))For each row of the left table, pandas looks up the row of orders with the same order_id and copies its columns across. Many item lines point to one order, so the relationship is many to one, and validate="many_to_one" tells pandas to check that: if orders had two rows for one id, the merge would raise MergeError instead of quietly doubling those items. The row count before and after is the second check. A left merge onto a unique key never changes it.
Now revenue can finally respect the business rule you postponed in Lesson 2.5: cancelled orders are not revenue.
\nimport pandas as pd\n\nitems = pd.read_csv(\"order_items.csv\")\norders = pd.read_csv(\"orders.csv\", parse_dates=[\"order_date\"])\nitems[\"revenue\"] = items[\"quantity\"] * items[\"unit_price\"] * (1 - items[\"discount\"])\nlines = items.merge(orders, on=\"order_id\", how=\"left\", validate=\"many_to_one\")\n\nkept = lines[lines[\"status\"] != \"cancelled\"]\nprint(round(kept[\"revenue\"].sum(), 2), \"revenue excluding cancelled orders\")446,666.32 dollars. This is the revenue definition for the rest of the course: line revenue after discounts, excluding shipping fees and cancelled orders. Returned orders stay in, because their refunds are tracked separately in returns. Write definitions like this down where readers will see them; a number without its definition invites an argument.
When the key has a different name on each side, name both: left_on="customer_id", right_on="referred_by" in a merge of customers with itself matches each customer to the people they referred. And pick the key that identifies what you are matching. Returns belong to order lines, so returns joins order_items on order_item_id; joining it on order_id instead would attach each refund to every line of the order.
how decides what happens to keys that exist on only one side:
inner (the default) keeps only keys found on both sides. left keeps every left row, filling the right-hand columns with NaN where there is no partner. right is the mirror image, and outer keeps everything. For adding information to a table of facts, such as orders, items or sessions, use how="left" with the fact table on the left: you never lose a fact because a lookup table is incomplete, and the NaNs show you exactly where it is.
indicator=True adds a _merge column recording where each row came from:
import pandas as pd\n\ncustomers = pd.read_csv(\"customers.csv\")\norders = pd.read_csv(\"orders.csv\")\nbuyers = orders[[\"customer_id\"]].drop_duplicates()\n\ncheck = customers.merge(buyers, on=\"customer_id\", how=\"left\", indicator=True)\nprint(check[\"_merge\"].value_counts())\nnever_ordered = check[check[\"_merge\"] == \"left_only\"]\nprint(len(never_ordered), \"customers never ordered\")85 customers signed up and never bought anything, the same number you found with nunique in Lesson 2.2, now as a list of names you could hand to the marketing team. indicator=True is also the fastest audit of any merge: right_only rows in a left join are impossible, and a surprising number of left_only rows means keys that should match do not, often because of types (int on one side, text on the other) or untrimmed spaces.
order_items and products both have a unit_price column: the price charged and the list price. pandas adds suffixes to tell them apart, and you can choose them:
import pandas as pd\n\nitems = pd.read_csv(\"order_items.csv\")\norders = pd.read_csv(\"orders.csv\", parse_dates=[\"order_date\"])\nproducts = pd.read_csv(\"products.csv\")\n\nlines = (\n items.merge(orders[[\"order_id\", \"order_date\"]], on=\"order_id\", validate=\"many_to_one\")\n .merge(products[[\"product_id\", \"name\", \"unit_price\"]], on=\"product_id\",\n suffixes=(\"_charged\", \"_list\"), validate=\"many_to_one\")\n)\nlines[\"price_ratio\"] = (lines[\"unit_price_charged\"] / lines[\"unit_price_list\"]).round(2)\nprint(lines.groupby(lines[\"order_date\"] >= \"2025-03-01\")[\"price_ratio\"].agg([\"min\", \"max\"]))Before March 2025, every line was charged exactly the list price; from 1 March, every line was charged 5% more. The merge has just confirmed the price rise from the data alone, and it is one of the suspects for the Q4 revenue jump.
\n\nIn the exercise you build the full line-level table that the rest of the course analyses. Next you reshape tables between wide and long formats with pivot_table and melt.
left.merge(right, on="key", how=...) combines tables by matching key values; how decides what happens to rows without a match.","how="left" keeps every row of the left table and is the safe default for adding columns to a fact table.","Pass validate="many_to_one" (or "one_to_one") so pandas raises MergeError if the right-hand keys are not unique, instead of silently multiplying rows.","Compare len() before and after every merge; a left merge onto a unique key never changes the row count.","indicator=True adds a _merge column (both, left_only, right_only) that shows which rows found a partner."],"further":[{"title":"Merge, join, concatenate and compare","url":"https://pandas.pydata.org/docs/user_guide/merging.html"},{"title":"pandas.DataFrame.merge","url":"https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.merge.html"}],"quiz":[{"q":"You left-merge order_items (3,991 rows) with orders on order_id. How many rows should the result have?
how="left", keeps the left table’s row count. validate="many_to_one" guarantees it."}],"answer":3},{"q":"Which merge finds the customers who never placed an order?
","options":[{"text":"customers.merge(orders, on="customer_id", how="inner")","why":"An inner join keeps only customers with at least one order, which is the opposite of what you want."},{"text":"customers.merge(orders, on="customer_id", how="left", indicator=True), then keep _merge == "left_only"","why":"Correct. A left join keeps every customer, and left_only marks those with no matching order. That finds 85 customers."},{"text":"orders.merge(customers, on="customer_id", how="left")","why":"With orders on the left, every row is an order, so customers without orders never appear."}],"answer":1},{"q":"orders.merge(tickets, on="customer_id") returns 6,624 rows from 2,066 orders and 873 tickets. What happened?
customer_id repeats on both sides, so every order of a customer was paired with every ticket of that customer.","why":"Correct. This is a many-to-many join. validate="many_to_one" would have raised an error; join tickets to orders on order_id instead."},{"text":"pandas added blank rows for tickets without orders.","why":"The default how="inner" adds no unmatched rows at all."},{"text":"The merge appended the two tables one below the other.","why":"That is pd.concat. A merge matches rows side by side."}],"answer":0}],"exercise":{"kind":"python","files":["order_items.csv","orders.csv","products.csv","customers.csv"],"starter":"import pandas as pd\n\nitems = pd.read_csv(\"order_items.csv\")\norders = pd.read_csv(\"orders.csv\", parse_dates=[\"order_date\"])\nproducts = pd.read_csv(\"products.csv\")\ncustomers = pd.read_csv(\"customers.csv\")\nitems[\"revenue\"] = items[\"quantity\"] * items[\"unit_price\"] * (1 - items[\"discount\"])\n\n# 1. sales: every item line with its order's columns, the product name (as \"product\"),\n# and the customer's country. Use left merges with validate=\"many_to_one\".\n# Then drop the lines of cancelled orders.\nsales = None\n\n# 2. Revenue by country, a Series sorted from highest to lowest\nrevenue_by_country = None\n","solution":"import pandas as pd\n\nitems = pd.read_csv(\"order_items.csv\")\norders = pd.read_csv(\"orders.csv\", parse_dates=[\"order_date\"])\nproducts = pd.read_csv(\"products.csv\")\ncustomers = pd.read_csv(\"customers.csv\")\nitems[\"revenue\"] = items[\"quantity\"] * items[\"unit_price\"] * (1 - items[\"discount\"])\n\nsales = (\n items.merge(orders, on=\"order_id\", how=\"left\", validate=\"many_to_one\")\n .merge(products[[\"product_id\", \"name\"]].rename(columns={\"name\": \"product\"}),\n on=\"product_id\", how=\"left\", validate=\"many_to_one\")\n .merge(customers[[\"customer_id\", \"country\"]], on=\"customer_id\", how=\"left\", validate=\"many_to_one\")\n)\nprint(len(sales))\nsales = sales[sales[\"status\"] != \"cancelled\"]\n\nrevenue_by_country = sales.groupby(\"country\")[\"revenue\"].sum().sort_values(ascending=False)\nprint(revenue_by_country.round(2))\n","checks":[{"id":"columns","test":"isinstance(sales, pd.DataFrame) and {'revenue', 'order_date', 'status', 'product', 'country'} <= set(sales.columns)"},{"id":"no-cancelled","test":"(sales['status'] != 'cancelled').all()"},{"id":"row-count","test":"len(sales) == int((items['order_id'].map(orders.set_index('order_id')['status']) != 'cancelled').sum())"},{"id":"revenue-total","test":"round(float(sales['revenue'].sum()), 2) == 446666.32"},{"id":"by-country","test":"isinstance(revenue_by_country, pd.Series) and revenue_by_country.index[0] == 'United Arab Emirates' and abs(float(revenue_by_country.iloc[0]) - 74531.8) < 0.01 and revenue_by_country.is_monotonic_decreasing"}],"title":"Build the sales table","prompt":"Build sales: every row of items, plus its order’s columns from orders, the product’s name renamed to product, and the customer’s country. Use left merges with validate="many_to_one", then remove the lines of cancelled orders. Then set revenue_by_country to total revenue per country as a Series, sorted from highest to lowest.
items.merge(orders, on="order_id", how="left", validate="many_to_one"), then products on product_id, then customers on customer_id.","Select only the columns you need from the lookup tables, e.g. products[["product_id", "name"]].rename(columns={"name": "product"}), so you don’t import clashing names like unit_price.","Filter with sales[sales["status"] != "cancelled"], then groupby("country")["revenue"].sum().sort_values(ascending=False)."],"explanation":"Each merge adds columns without changing the number of rows, because every item has exactly one order, one product and one customer, which validate="many_to_one" guarantees. Selecting only the lookup columns you need keeps the table tidy and avoids _x/_y suffixes. After removing cancelled orders the revenue is 446,666.32 dollars, the course’s revenue definition. The UAE leads with 74,531.80 dollars, just ahead of the UK, even though the US has more customers: a first sign that revenue per customer differs by country.
sales has revenue, order, product and country columns","no-cancelled":"No cancelled orders remain","row-count":"Every non-cancelled item line is present exactly once","revenue-total":"Total revenue is 446,666.32","by-country":"revenue_by_country is sorted, with the UAE first at 74,531.80"}},"words":693,"readMinutes":3}