Descriptive Statistics and Distributions

Someone asks: “What’s a typical Cartwheel order?” You compute the mean, 228 dollars, and say so. Then the marketing team designs a free-shipping threshold around 228 dollars, and discovers that two thirds of orders never reach it. The mean was correct and the answer was wrong, because order values are not spread evenly around their average. Describing data honestly means describing its shape, not only its centre.

Mean, median and quantiles

import pandas as pd

items = pd.read_csv("order_items.csv")
orders = pd.read_csv("orders.csv", parse_dates=["order_date"])
items["revenue"] = items["quantity"] * items["unit_price"] * (1 - items["discount"])
orders = orders.merge(items.groupby("order_id", as_index=False)["revenue"].sum(), on="order_id", validate="one_to_one")
orders = orders[orders["status"] != "cancelled"]
value = orders["revenue"]

print(value.describe(percentiles=[0.1, 0.25, 0.5, 0.75, 0.9, 0.99]).round(2))
print("skew:", round(value.skew(), 2))

Read it from the middle out. The median (50%) is 147 dollars: half of all orders are smaller. The quartiles say the middle half of orders falls between 68 and 303 dollars. The 90th percentile is 505 and the 99th is 1,168, while the maximum is 2,117. The mean, 228, sits far above the median because of that long right tail, and the positive skew of 2.43 confirms it. For a typical value in skewed data, quote the median; for totals and forecasts, the mean is the right number, because mean × count = total.

std, the standard deviation, is 241 dollars, larger than the median itself. That is another sign of a spread-out, skewed distribution; in that situation quantiles describe the spread far better than “mean plus or minus one standard deviation”.

The shape, without a chart

value_counts with bins counts how many values fall in each range, which is a histogram in text form:

import pandas as pd

items = pd.read_csv("order_items.csv")
orders = pd.read_csv("orders.csv", parse_dates=["order_date"])
items["revenue"] = items["quantity"] * items["unit_price"] * (1 - items["discount"])
orders = orders.merge(items.groupby("order_id", as_index=False)["revenue"].sum(), on="order_id", validate="one_to_one")
orders = orders[orders["status"] != "cancelled"]

print(orders["revenue"].value_counts(bins=[0, 50, 100, 200, 400, 800, 3000]).sort_index())

Most orders sit between 50 and 400 dollars, and a thinning tail runs past 800. Notice the bands get wider as they go right; equal-width bands of 100 dollars would squash nearly everything into the first few bars.

Outliers: flag, then decide

A common rule of thumb flags values more than 1.5 interquartile ranges (IQR, the distance between the quartiles) beyond the quartiles:

import pandas as pd

items = pd.read_csv("order_items.csv")
orders = pd.read_csv("orders.csv", parse_dates=["order_date"])
items["revenue"] = items["quantity"] * items["unit_price"] * (1 - items["discount"])
orders = orders.merge(items.groupby("order_id", as_index=False)["revenue"].sum(), on="order_id", validate="one_to_one")
orders = orders[orders["status"] != "cancelled"]
value = orders["revenue"]

q1, q3 = value.quantile([0.25, 0.75])
upper = q3 + 1.5 * (q3 - q1)
big = value > upper
print(f"fence {upper:.2f}: {big.sum()} orders, {value[big].sum() / value.sum():.1%} of revenue")

125 orders are above 656.62 dollars, and together they bring in 25.9% of revenue. These are not errors to clean away: they are multi-item orders of desks, chairs and dumbbells, real and important. An outlier rule tells you where to look. Data entry errors, such as a total with a missing decimal point, get fixed; genuine extremes stay, and you check whether your conclusion survives with and without them.

The same lens over time

Distributions can be compared across periods too, and this one carries a clue for the case study. If Q4 2025 revenue jumped because customers bought bigger baskets, the typical order should have grown:

import pandas as pd

items = pd.read_csv("order_items.csv")
orders = pd.read_csv("orders.csv", parse_dates=["order_date"])
items["revenue"] = items["quantity"] * items["unit_price"] * (1 - items["discount"])
orders = orders.merge(items.groupby("order_id", as_index=False)["revenue"].sum(), on="order_id", validate="one_to_one")
orders = orders[orders["status"] != "cancelled"]

quarter = orders["order_date"].dt.to_period("Q")
print(orders.groupby(quarter)["revenue"].agg(["count", "median"]).round(1).tail(4))

It did not. The median order in Q4 2025 was 151.70 dollars, slightly below Q3’s 159.20, while the number of orders doubled. Whatever happened in Q4, it was about how many orders arrived, not how big they were. One table of medians has already ruled out a popular explanation.

Correlation: moving together is not causing

corr() measures how strongly two numeric columns move together, from -1 (perfectly opposite) through 0 (no linear relation) to 1 (perfectly together). churn.csv has one row per customer and a churned flag:

import pandas as pd

churn = pd.read_csv("churn.csv")
cols = ["tenure_days", "orders", "days_since_last_order", "support_tickets", "churned"]
print(churn[cols].corr().round(2)["churned"])
print(churn.groupby("churned")["days_since_last_order"].median())

Customers who churned had gone far longer since their last order: a median of 214 days against 46, and a correlation of 0.45. That makes the gap a good warning signal. It does not show that the gap causes churn; a customer who has already lost interest stops ordering and churns, and both measurements reflect that one fact. Treat correlations as leads to investigate, and keep the word “causes” for experiments.

In the exercise you describe Cartwheel’s order values yourself. Next you turn numbers like these into charts.