“What’s our average order value?” sounds like one number, but order_items has no order value column: an order with four products has four rows. You first need one row per order, with the order’s revenue summed from its lines. “Revenue per product”, “orders per channel” and “sales per country” are the same shape of question. In a spreadsheet you would build a pivot table; in pandas you use groupby, the method you will call more than any other.
import pandas as pd\n\nitems = pd.read_csv(\"order_items.csv\")\nitems[\"revenue\"] = items[\"quantity\"] * items[\"unit_price\"] * (1 - items[\"discount\"])\n\nby_product = items.groupby(\"product_id\")[\"revenue\"].sum()\nprint(by_product.sort_values(ascending=False).head(3).round(2))\nprint(round(by_product.sum(), 2), round(items[\"revenue\"].sum(), 2))Read the middle line as three steps. Split the rows into groups, one per product_id. Apply a calculation, sum, to the revenue column of each group. Combine the results into a Series indexed by the group key. Product 21, the Standing Desk, earned 60,017.83 dollars, almost twice the next product.
The last line is a habit worth copying: the group totals add up to the overall total. If they do not, rows went missing (for example, rows whose key is NaN, which groupby drops by default) or were duplicated before you grouped.
One number per group is rarely enough. agg with named aggregation computes several, and names each output column:
import pandas as pd\n\nitems = pd.read_csv(\"order_items.csv\")\nitems[\"revenue\"] = items[\"quantity\"] * items[\"unit_price\"] * (1 - items[\"discount\"])\n\nper_order = items.groupby(\"order_id\").agg(\n revenue=(\"revenue\", \"sum\"),\n lines=(\"order_item_id\", \"count\"),\n units=(\"quantity\", \"sum\"),\n max_discount=(\"discount\", \"max\"),\n)\nprint(per_order.head().round(2))\nprint(per_order.shape, \"| average order value:\", round(per_order[\"revenue\"].mean(), 2))Each keyword is an output column, and each value is a pair: (input column, function). Functions are given by name as strings: "sum", "mean", "median", "min", "max", "count", "size", "nunique", "first", "last", "std". The result has one row per order, 2,066 of them, and the average order value is 228.43 dollars, across all statuses, cancellations included, for now.
You will also see the older style agg({"revenue": "sum", "discount": "max"}), or agg(["sum", "mean"]) on one column. They work, but they reuse the input names, so a table with two statistics of the same column ends up with confusing headers. Named aggregation says exactly what each column is.
Two ways to count, and they answer different questions:
\nimport pandas as pd\n\norders = pd.read_csv(\"orders.csv\")\nprint(orders.groupby(\"channel\").size()) # rows per group\nprint(orders.groupby(\"channel\")[\"coupon_code\"].count()) # non-missing coupons per groupsize() counts rows: 1,146 web orders. count() counts the non-missing values of a column: 208 of those web orders have a coupon code. Use size (or count on a column that is never missing, such as an id) for “how many”, and count for “how many have a value”.
Group by a list of columns to cross two dimensions. The result has a two-level index (a MultiIndex), and unstack() spreads the inner level into columns, which reads like a pivot table:
import pandas as pd\n\norders = pd.read_csv(\"orders.csv\")\nby_channel_status = orders.groupby([\"channel\", \"status\"]).size()\nprint(by_channel_status.head(4))\nprint(by_channel_status.unstack())\n\nflat = orders.groupby(\"channel\", as_index=False).agg(\n orders=(\"order_id\", \"count\"),\n avg_fee=(\"shipping_fee\", \"mean\"),\n)\nprint(flat.round(2))as_index=False keeps the group keys as ordinary columns instead of moving them into the index, which is handy when the result feeds a merge, a chart or a CSV export. reset_index() on an indexed result does the same afterwards.
Sometimes you want the group’s number next to each row rather than one row per group. “What share of its order does each line represent?” needs every line to know its order’s total. transform runs the same group calculation but returns a result aligned to the original rows:
import pandas as pd\n\nitems = pd.read_csv(\"order_items.csv\")\nitems[\"revenue\"] = items[\"quantity\"] * items[\"unit_price\"] * (1 - items[\"discount\"])\nitems[\"order_revenue\"] = items.groupby(\"order_id\")[\"revenue\"].transform(\"sum\")\nitems[\"share_of_order\"] = items[\"revenue\"] / items[\"order_revenue\"]\nprint(items[[\"order_id\", \"revenue\", \"order_revenue\", \"share_of_order\"]].head(5).round(3))Order 1001 has four lines, and the Burr Coffee Grinder line carries 62% of its value. The rule of thumb: agg when you want a summary table, transform when you want a new column on the detailed table.
The exercise builds the per-order table you will use from here on. Next you combine it with orders and customers, because the questions that matter, such as “revenue by country, excluding cancellations”, need columns from more than one table.
groupby splits rows into groups by key, applies a calculation to each group, and combines the results into one table.","Named aggregation, agg(revenue=("revenue", "sum"), lines=("order_item_id", "count")), gives each output column a clear name.","size() counts rows per group; count() counts non-missing values in a column, so the two differ when the column has gaps.","Grouping by several keys gives a MultiIndex; as_index=False or reset_index() turns the keys back into ordinary columns.","Check that group results add back up to the overall total, so you know no rows were lost or double-counted."],"further":[{"title":"Group by: split-apply-combine","url":"https://pandas.pydata.org/docs/user_guide/groupby.html"},{"title":"pandas.core.groupby.DataFrameGroupBy.agg","url":"https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.DataFrameGroupBy.agg.html"}],"quiz":[{"q":"orders.groupby("channel")["coupon_code"].count() gives 208 for web, but orders.groupby("channel").size() gives 1,146. Why?
count() counts only non-missing coupon codes, while size() counts every row in the group.","why":"Correct. 208 web orders used a coupon; 1,146 web orders exist. Use size() to count rows and count() to count filled values."},{"text":"count() counts distinct coupon codes.","why":"Distinct values are nunique(). count() counts every non-missing value, repeats included."},{"text":"size() double-counts orders with several items.","why":"orders has one row per order, so there is nothing to double-count."}],"answer":1},{"q":"Which call returns one row per order with columns named revenue and lines?
items.groupby("order_id").agg(revenue=("revenue", "sum"), lines=("order_item_id", "count"))","why":"Correct. Each keyword becomes an output column, defined by a (column, function) pair."},{"text":"items.groupby("order_id")["revenue", "order_item_id"].sum()","why":"Selecting several columns needs a list (double brackets), and even then you get two sums with the original names, not a count."},{"text":"items.agg(revenue=("revenue", "sum"), lines=("order_item_id", "count"))","why":"Without groupby this aggregates the whole table into single totals, not one row per order."}],"answer":0},{"q":"You group items by order and sum revenue, then sum that result. The grand total is 3% higher than items["revenue"].sum(). What is the most likely explanation?
Using items (revenue column already added), build per_order: one row per order_id with the columns revenue (sum), lines (number of order lines) and units (sum of quantity). Then set avg_order_value to the mean order revenue rounded to 2 decimals, multi_line_share to the share of orders with 2 or more lines rounded to 3 decimals, and top_product to the product_id with the highest total revenue.
items.groupby("order_id").agg(revenue=("revenue", "sum"), ...) with one keyword per output column.","Count lines with ("order_item_id", "count") or ("order_id", "size"). A share is the mean of a boolean: (per_order["lines"] >= 2).mean().","items.groupby("product_id")["revenue"].sum().idxmax() returns the label of the largest group."],"explanation":"Named aggregation turns 3,991 lines into 2,066 orders in one call, with readable column names. The checks confirm the parts add up: the lines column sums back to 3,991 and units to the total quantity, so no line was lost or counted twice. Average order value is 228.43 dollars, and 55.1% of orders contain more than one product, a useful fact when someone proposes a promotion on single items. Product 21, the Standing Desk 140cm, is the top earner. as_index=False would also be accepted: then order_id is a column instead of the index.
per_order has 2,066 rows and the columns revenue, lines, units","order-1001":"Order 1001 has revenue 438.90","lines-units":"Lines and units add back up to the item table’s totals","aov":"avg_order_value is 228.43","multi":"multi_line_share is 0.551","top":"top_product is 21"}},"words":690,"readMinutes":3}