{"id":"l-ml-5-4","courseId":"ml-crash-course","lang":"en","kind":"lesson","summary":"Audit a model’s errors group by group, pick a fairness criterion that matches the harm, and monitor inputs, predictions and outcomes so you notice when the world drifts away from the training data.","html":"

Your champion model reached a test ROC AUC of 0.81. That’s one number for 100 customers from eight countries and three segments. It’s entirely possible for a model to be good on average and poor for one group, and for that group to be the one with the least power to complain. It’s also possible for a model that was good in October to be quietly wrong by March. This lesson covers both: fairness before launch, drift after it.

\n

Where bias comes from

\n

Bias in a model is rarely a line of code. It arrives with the data and with how predictions are used:

\n\n

Auditing a model group by group

\n

To audit fairly you need predictions for many customers that the model didn’t train on. cross_val_predict gives every row an out-of-fold prediction: each customer is scored by a model trained on the other folds.

\n
Python
import pandas as pd\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.metrics import precision_score, recall_score\nfrom sklearn.model_selection import StratifiedKFold, cross_val_predict\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.preprocessing import StandardScaler\n\nchurn = pd.read_csv(\"churn.csv\")\nfeatures = [\"tenure_days\", \"orders\", \"total_spent\", \"avg_order_value\",\n            \"days_since_last_order\", \"used_coupon\", \"support_tickets\", \"mobile_share\"]\ncv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)\nmodel = make_pipeline(StandardScaler(), LogisticRegression())\nprob = cross_val_predict(model, churn[features], churn[\"churned\"], cv=cv, method=\"predict_proba\")[:, 1]\naudit = churn.assign(flagged=(prob >= 0.5).astype(int))\n\ndef group_report(g):\n    return pd.Series({\n        \"customers\": len(g),\n        \"churn_rate\": g[\"churned\"].mean(),\n        \"flag_rate\": g[\"flagged\"].mean(),\n        \"recall\": recall_score(g[\"churned\"], g[\"flagged\"]),\n        \"precision\": precision_score(g[\"churned\"], g[\"flagged\"]),\n    })\n\nprint(audit.groupby(\"segment\")[[\"churned\", \"flagged\"]].apply(group_report).round(2))\nprint(audit.groupby(\"country\")[[\"churned\", \"flagged\"]].apply(group_report).round(2))
\n

Read across the columns:

\n\n

Recall ranges from 0.71 (Canada, UAE, UK) to 0.82 (Egypt, Jordan). Precision for small businesses is 0.66 against 0.78 for consumers. Some gaps are noise: Jordan’s numbers rest on 17 customers, so a single customer moves them by several points. Others deserve a closer look as data grows. The point of the audit isn’t a pass/fail stamp; it’s knowing where the model is weaker, so you can decide whether that’s acceptable.

\n

Choosing what “fair” means

\n

There are several reasonable definitions, and when groups have different base rates (Egypt churns at 67%, Germany at 45%) they can’t all hold at once:

\n\n

Pick by asking who is harmed by which error. For win-back offers the harm is a churner who never gets the offer, so equal recall is the natural target. For a model that freezes accounts, the harm is a false positive, and you’d look at false positive rates instead. Write the choice down; it’s a business and ethical decision, not a technical default.

\n\n

After launch: drift

\n

The model learned from customers as they were on 30 September 2025. The world moves. Data drift is a change in the inputs: a new mobile app shifts mobile_share, a courier strike stretches days_since_last_order. Concept drift is a change in the relationship itself: a competitor’s loyalty scheme makes even recent customers leave. And churn has label delay: you only learn whether a prediction was right 90 days later.

\n
\n A loop with four stages. Daily: check input distributions against training data. Daily: track the share of customers flagged. After 90 days: compare predictions with real outcomes. When checks fail: investigate, then retrain and re-audit, which feeds back to the deployed model.\n \n deployed model\n \n input drift checks\n daily, no labels needed\n \n prediction mix\n flag rate per group\n \n real performance\n after 90 days\n \n investigate, retrain, re-audit\n \n \n \n \n \n
Monitoring a deployed churn model
\n

Input drift can be checked every day, without labels, by comparing each feature’s recent values with the training data. The two-sample Kolmogorov–Smirnov test from SciPy is a simple start:

\n
Python
import numpy as np\nimport pandas as pd\nfrom scipy.stats import ks_2samp\n\nchurn = pd.read_csv(\"churn.csv\")\nfeatures = [\"tenure_days\", \"orders\", \"total_spent\", \"avg_order_value\",\n            \"days_since_last_order\", \"used_coupon\", \"support_tickets\", \"mobile_share\"]\ntraining = churn[features]\n\n# Simulate next quarter: a courier problem delays orders and a new app shifts shopping to mobile\nrng = np.random.default_rng(7)\nrecent = training.sample(150, random_state=1).copy()\nrecent[\"days_since_last_order\"] += rng.integers(20, 60, len(recent))\nrecent[\"mobile_share\"] = (recent[\"mobile_share\"] + 0.3).clip(0, 1)\n\nfor col in features:\n    stat, p = ks_2samp(training[col], recent[col])\n    flag = \"DRIFT\" if p < 0.01 else \"\"\n    print(f\"{col:22} KS {stat:.2f}  p={p:.3g} {flag}\")
\n

The two columns that changed light up; the rest don’t. In production you’d run this on a schedule, alert on sustained drift rather than a single bad day, and pair it with the flag rate per group and, once outcomes arrive, recall and precision per group. When real performance drops, retrain on recent data and rerun this lesson’s audit before redeploying.

\n\n

Monitoring assumes the model is running somewhere. The last lesson packages the champion pipeline, puts it behind an API and closes the loop.

\n","toc":[{"id":"sec-1","text":"Where bias comes from","level":2},{"id":"sec-2","text":"Auditing a model group by group","level":2},{"id":"sec-3","text":"Choosing what “fair” means","level":2},{"id":"sec-4","text":"After launch: drift","level":2}],"takeaways":["Bias enters through the data (historical labels, under-represented groups, proxy features) and through how predictions are used; a good overall score can hide a bad score for one group.","Audit with per-group metrics: selection rate, recall and precision, computed on out-of-fold or test predictions.","Fairness definitions such as equal selection rates, equal recall and equal precision usually can’t all hold at once; choose the one that matches who is harmed by which error.","Dropping a sensitive column doesn’t remove bias, because other features act as proxies; keep it available for auditing.","Monitor three things after launch: input distributions (data drift), the distribution of predictions, and real performance once outcomes arrive."],"further":[{"title":"cross_val_predict","url":"https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_val_predict.html"},{"title":"Model evaluation: classification metrics","url":"https://scikit-learn.org/stable/modules/model_evaluation.html#classification-metrics"}],"quiz":[{"q":"

The churn model’s recall is 0.82 for Egyptian customers and 0.71 for Canadian customers. Win-back offers go to flagged customers. Who is disadvantaged?

","options":[{"text":"Canadian churners, who are less likely to be flagged and offered a deal.","why":"Correct. Lower recall means more Canadian churners are missed, so fewer of them receive the benefit."},{"text":"Egyptian customers, because more of them are flagged.","why":"Being flagged here means receiving an offer, which is a benefit, not a penalty."},{"text":"Nobody, because the overall recall is fine.","why":"An overall average can hide a group that’s consistently served worse."},{"text":"Canadian customers who stayed, because they’re flagged more.","why":"That would be about false positives (precision or false positive rate), not recall."}],"answer":0},{"q":"

A teammate removes country from the features ‘to make the model fair’. Why is that not enough?

","options":[{"text":"Removing a column always lowers accuracy below the baseline.","why":"It might change accuracy a little or not at all; the fairness problem is separate."},{"text":"Other features, such as shipping fees or delivery times, can carry the same information, and you can no longer audit by country.","why":"Correct. Proxies let the bias back in, and without the column you can’t measure whether it did."},{"text":"scikit-learn requires sensitive attributes to stay in the model.","why":"There’s no such requirement; the issue is about proxies and auditing."}],"answer":1},{"q":"

Two months after launch, the share of customers flagged as churners rises from 47% to 60%, but no churn outcomes are known yet. What’s the best first step?

","options":[{"text":"Retrain immediately on the latest data.","why":"You don’t have new labels yet, and you don’t know whether the model or the world changed."},{"text":"Lower the threshold so the flag rate returns to 47%.","why":"That hides the signal instead of explaining it."},{"text":"Ignore it until the 90-day outcomes arrive.","why":"Waiting three months for a problem you can investigate today is costly."},{"text":"Compare current input distributions with the training data to find which features shifted, then ask the business why.","why":"Correct. Data drift checks work without labels and usually point to a cause, such as a delivery problem or a new app."}],"answer":3}],"exercise":{"kind":"python","files":["churn.csv"],"starter":"import numpy as np\nimport pandas as pd\nfrom scipy.stats import ks_2samp\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.metrics import precision_score, recall_score\nfrom sklearn.model_selection import StratifiedKFold, cross_val_predict\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.preprocessing import StandardScaler\n\nchurn = pd.read_csv(\"churn.csv\")\nfeatures = [\"tenure_days\", \"orders\", \"total_spent\", \"avg_order_value\",\n \"days_since_last_order\", \"used_coupon\", \"support_tickets\", \"mobile_share\"]\ncv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)\nmodel = make_pipeline(StandardScaler(), LogisticRegression())\n\n# Next quarter's customers (simulated): a premium range lifted basket sizes and a returns policy change caused more tickets\nrng = np.random.default_rng(3)\nrecent = churn[features].sample(150, random_state=2).copy()\nrecent[\"avg_order_value\"] *= 1.6\nrecent[\"support_tickets\"] += rng.integers(1, 3, len(recent))\n\n# 1. Out-of-fold churn flags (threshold 0.5) for every customer, using cross_val_predict\nflagged = None\n\n# 2. Recall per segment (a Series indexed by segment), and the segment with the lowest recall\nrecall_by_segment = None\nlowest_recall_segment = None\n\n# 3. Columns of `recent` whose distribution differs from churn[features] (KS test p < 0.01), as a list\ndrifted = []\n","solution":"import numpy as np\nimport pandas as pd\nfrom scipy.stats import ks_2samp\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.metrics import precision_score, recall_score\nfrom sklearn.model_selection import StratifiedKFold, cross_val_predict\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.preprocessing import StandardScaler\n\nchurn = pd.read_csv(\"churn.csv\")\nfeatures = [\"tenure_days\", \"orders\", \"total_spent\", \"avg_order_value\",\n \"days_since_last_order\", \"used_coupon\", \"support_tickets\", \"mobile_share\"]\ncv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)\nmodel = make_pipeline(StandardScaler(), LogisticRegression())\n\n# Next quarter's customers (simulated): a premium range lifted basket sizes and a returns policy change caused more tickets\nrng = np.random.default_rng(3)\nrecent = churn[features].sample(150, random_state=2).copy()\nrecent[\"avg_order_value\"] *= 1.6\nrecent[\"support_tickets\"] += rng.integers(1, 3, len(recent))\n\nprob = cross_val_predict(model, churn[features], churn[\"churned\"], cv=cv, method=\"predict_proba\")[:, 1]\nflagged = (prob >= 0.5).astype(int)\n\naudit = churn.assign(flagged=flagged)\nrecall_by_segment = audit.groupby(\"segment\")[[\"churned\", \"flagged\"]].apply(\n lambda g: recall_score(g[\"churned\"], g[\"flagged\"]))\nlowest_recall_segment = recall_by_segment.idxmin()\n\ndrifted = [c for c in features if ks_2samp(churn[c], recent[c]).pvalue < 0.01]\nprint(recall_by_segment.round(2).to_dict(), lowest_recall_segment, drifted)\n","checks":[{"id":"flags","test":"flagged is not None and len(flagged) == 398 and set(np.unique(flagged)) <= {0, 1} and abs(int(np.sum(flagged)) - 218) <= 2"},{"id":"recall","test":"recall_by_segment is not None and set(recall_by_segment.index) == {'consumer', 'enterprise', 'small_business'} and abs(float(recall_by_segment['consumer']) - 0.7582) < 0.01"},{"id":"lowest","test":"lowest_recall_segment == 'enterprise'"},{"id":"drift","test":"sorted(drifted) == ['avg_order_value', 'support_tickets']"}],"title":"Audit by segment, then check for drift","prompt":"
    \n
  1. flagged: an out-of-fold 0/1 churn flag for every customer, from cross_val_predict(model, ..., cv=cv, method="predict_proba") and a 0.5 threshold.
    2. \n
  2. recall_by_segment: recall of those flags within each segment (a Series indexed by segment). lowest_recall_segment: the segment whose churners are caught least often.
    3. \n
  3. drifted: the list of feature names where recent differs from churn[features] according to a two-sample KS test with p < 0.01.
    4. \n
\n","hints":["cross_val_predict(...)[:, 1] gives the churn probability column; compare it with 0.5 and convert with .astype(int).","Add the flags as a column, then audit.groupby("segment")[["churned", "flagged"]].apply(lambda g: recall_score(g["churned"], g["flagged"])).","ks_2samp(churn[c], recent[c]).pvalue for each c in features; keep those below 0.01."],"explanation":"

Out-of-fold predictions let you audit all 398 customers without scoring anyone with a model that trained on them. Enterprise churners are caught least often (about 0.71, against 0.76 for consumers), but with only 24 enterprise customers that gap needs more data before anyone acts on it. The KS checks flag exactly the two columns the simulation changed: average order value and support tickets. That is the early warning you’d get months before the churn outcomes confirm a drop in performance.

\n","checkLabels":{"flags":"flagged holds out-of-fold 0/1 flags for all 398 customers","recall":"recall_by_segment has the recall of each segment","lowest":"lowest_recall_segment names the weakest segment","drift":"drifted lists exactly the shifted columns"}},"words":818,"readMinutes":4}