Ask a support lead how they spot customers who are about to leave and you’ll get something like: “If they haven’t ordered in months, they’re gone. If they’re recent but only bought once or twice, it’s a toss-up. Regulars who ordered recently are fine.” That’s a decision tree. This lesson grows one from data and checks whether it agrees.
\nA decision tree predicts by asking yes/no questions about one feature at a time. To build it, the algorithm looks at every feature and every possible threshold, and picks the question that splits the training customers into the two purest groups: ideally all churners on one side and all stayers on the other. Purity is measured with the Gini impurity by default (0 for a group of one class only, 0.5 for a 50/50 mix). Then it repeats the search inside each group, and so on down.
\nimport pandas as pd\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.tree import DecisionTreeClassifier, export_text\n\nchurn = pd.read_csv(\"churn.csv\")\nfeatures = [\"tenure_days\", \"orders\", \"total_spent\", \"avg_order_value\",\n \"days_since_last_order\", \"used_coupon\", \"support_tickets\", \"mobile_share\"]\nX_train, X_test, y_train, y_test = train_test_split(\n churn[features], churn[\"churned\"], test_size=0.25, random_state=42,\n stratify=churn[\"churned\"])\n\ntree = DecisionTreeClassifier(max_depth=2, random_state=42).fit(X_train, y_train)\nprint(export_text(tree, feature_names=features, show_weights=True))export_text prints the rules, and show_weights=True adds how many training customers of each class (stayed, churned) reached each leaf.
Three leaves match the support lead almost word for word. Long-silent customers churn (84 of 95). Recent regulars with four or more orders stay (60 of 73). Recent customers with few orders are a toss-up (67 of 127 churn). The tree also discovered an interaction nobody had to code: order count matters, but only among recent customers.
\nThe fourth leaf is a warning. The tree spent a whole question isolating three very long-tenured customers. That’s the greedy search at work: it picks the split that helps most right now, even if the help comes from a group too small to mean anything.
\nA customer who falls into a leaf gets that leaf’s class shares as probabilities. Land in the “regulars” leaf and predict_proba returns about [0.82, 0.18]. So a depth-2 tree can only ever output four distinct probabilities, one per leaf. That’s coarse, but honest. The trouble starts when the tree has no limits.
An unconstrained tree keeps splitting until every leaf is pure. On the churn data that’s 64 leaves, 11 levels deep, and 100% training accuracy. Every leaf is pure, so every probability is exactly 0 or 1: the tree claims total certainty about every test customer while being right only 72% of the time. Ranked by such probabilities, almost all customers tie, and the call list from last lesson becomes a lottery.
\nTwo hyperparameters do most of the work of keeping a tree honest:
\nmax_depth caps how many questions in a row the tree can ask.min_samples_leaf refuses any split that would leave fewer than that many training customers in a leaf. It targets exactly the problem in the fourth leaf above, and it lets the tree grow deep where there’s plenty of data and stay shallow where there isn’t.import pandas as pd\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.tree import DecisionTreeClassifier\n\nchurn = pd.read_csv(\"churn.csv\")\nfeatures = [\"tenure_days\", \"orders\", \"total_spent\", \"avg_order_value\",\n \"days_since_last_order\", \"used_coupon\", \"support_tickets\", \"mobile_share\"]\nX_train, X_test, y_train, y_test = train_test_split(\n churn[features], churn[\"churned\"], test_size=0.25, random_state=42,\n stratify=churn[\"churned\"])\n\nfor settings in [{}, {\"max_depth\": 3}, {\"min_samples_leaf\": 20}]:\n t = DecisionTreeClassifier(random_state=42, **settings).fit(X_train, y_train)\n distinct = len(set(t.predict_proba(X_test)[:, 1]))\n print(f\"{str(settings):24} leaves {t.get_n_leaves():3} train {t.score(X_train, y_train):.2f} \"\n f\"test {t.score(X_test, y_test):.2f} distinct probabilities {distinct}\")\n\nt = DecisionTreeClassifier(min_samples_leaf=20, random_state=42).fit(X_train, y_train)\nprint(pd.Series(t.feature_importances_, index=features).sort_values(ascending=False).round(2))feature_importances_ says how much each feature reduced impurity across all its splits, normalised to sum to 1. Recency takes over half, order count comes second; coupons and support tickets get zero because the tree never used them. Treat these numbers as a description of this tree, not of the world: features the tree didn’t need can still be informative, and the next lesson shows a more trustworthy way to measure importance.
That instability is also an opportunity. If one tree is noisy, average many trees grown on slightly different data and the noise cancels out. That’s the idea behind random forests and gradient boosting, next.
\n","toc":[{"id":"sec-1","text":"Twenty questions, chosen by data","level":2},{"id":"sec-2","text":"How a tree predicts probabilities","level":2},{"id":"sec-3","text":"Growing too far, and how to stop it","level":2}],"takeaways":["A decision tree splits the data with one yes/no question at a time, choosing at each step the question that makes the groups purest.","A fitted tree is a set of readable rules;export_text prints them.","Unconstrained trees grow until every leaf is pure, which memorises the training data and produces overconfident 0 or 1 probabilities.","max_depth and min_samples_leaf limit growth; min_samples_leaf stops the tree from spending a split on a handful of customers.","Trees need no feature scaling and capture interactions automatically, but a single tree is unstable: small data changes can reshape it."],"further":[{"title":"Decision Trees","url":"https://scikit-learn.org/stable/modules/tree.html"},{"title":"export_text","url":"https://scikit-learn.org/stable/modules/generated/sklearn.tree.export_text.html"}],"quiz":[{"q":"A leaf of a churn tree holds 60 customers who stayed and 13 who churned. What does predict_proba return for a customer who lands there?
[0.82, 0.18]: the class shares in that leaf.","why":"Correct. A tree’s probability is the fraction of training customers of each class in the leaf."},{"text":"[1.0, 0.0], because the leaf’s majority class is 0.","why":"That’s what predict reports as a label. predict_proba keeps the proportions."},{"text":"[0.5, 0.5], because trees don’t estimate probabilities.","why":"Trees do estimate them, from leaf proportions, though coarsely."},{"text":"It depends on how far the customer is from the split threshold.","why":"Inside a leaf every customer gets the same answer; distance from the threshold plays no part."}],"answer":0},{"q":"An unlimited tree’s test predictions contain only two distinct probabilities, 0.0 and 1.0. Why is that a problem?
","options":[{"text":"It means the tree has a bug and must be refitted with anotherrandom_state.","why":"It’s expected behaviour, not a bug. Every leaf of a fully grown tree is pure."},{"text":"Probabilities of exactly 0 or 1 can’t be used for ranking customers.","why":"They can be sorted, but with only two values almost every customer ties, so the ranking carries no information."},{"text":"The tree claims total certainty about every customer, which its test accuracy of 0.72 shows is false.","why":"Correct. Pure leaves produce overconfident probabilities, and the ranking you’d build from them is nearly useless."}],"answer":2},{"q":"Which is a genuine advantage of trees over logistic regression for the churn data?
","options":[{"text":"Trees always generalise better because they can model curves.","why":"Flexibility cuts both ways. On this data a constrained logistic regression does as well or better."},{"text":"Trees give smoother, better-calibrated probabilities.","why":"Tree probabilities are coarse leaf proportions, usually less smooth than logistic regression’s."},{"text":"Trees are stable; retraining on slightly different data gives nearly the same tree.","why":"The opposite is true. A different first split changes everything below it."},{"text":"Trees capture interactions such as “few orders matters only for recent customers” without you creating the feature.","why":"Correct. Each branch asks its own follow-up questions, so interactions come for free."}],"answer":3}],"exercise":{"kind":"python","files":["churn.csv"],"starter":"import pandas as pd\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.tree import DecisionTreeClassifier, export_text\n\nchurn = pd.read_csv(\"churn.csv\")\nfeatures = [\"tenure_days\", \"orders\", \"total_spent\", \"avg_order_value\",\n \"days_since_last_order\", \"used_coupon\", \"support_tickets\", \"mobile_share\"]\nX_train, X_test, y_train, y_test = train_test_split(\n churn[features], churn[\"churned\"], test_size=0.25, random_state=42,\n stratify=churn[\"churned\"])\n\ntree = DecisionTreeClassifier(max_depth=2, random_state=42).fit(X_train, y_train)\nprint(export_text(tree, feature_names=features))\n\n# 1. Rewrite the printed tree as plain Python rules. Return 1 for churn, 0 for stay.\ndef rule_predict(row):\n if row[\"days_since_last_order\"] <= 236.5:\n return 1\n return 1\n\nrule_preds = [rule_predict(row) for _, row in X_test.iterrows()]\n\n# 2. A tree limited by min_samples_leaf=20 (random_state=42), fitted on the training data\nleafy = None\n\n# 3. The name of the feature with the highest importance in `leafy`\ntop_feature = None\n","solution":"import pandas as pd\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.tree import DecisionTreeClassifier, export_text\n\nchurn = pd.read_csv(\"churn.csv\")\nfeatures = [\"tenure_days\", \"orders\", \"total_spent\", \"avg_order_value\",\n \"days_since_last_order\", \"used_coupon\", \"support_tickets\", \"mobile_share\"]\nX_train, X_test, y_train, y_test = train_test_split(\n churn[features], churn[\"churned\"], test_size=0.25, random_state=42,\n stratify=churn[\"churned\"])\n\ntree = DecisionTreeClassifier(max_depth=2, random_state=42).fit(X_train, y_train)\nprint(export_text(tree, feature_names=features))\n\ndef rule_predict(row):\n if row[\"days_since_last_order\"] <= 236.5:\n if row[\"orders\"] <= 3.5:\n return 1\n return 0\n if row[\"tenure_days\"] <= 637.5:\n return 1\n return 0\n\nrule_preds = [rule_predict(row) for _, row in X_test.iterrows()]\n\nleafy = DecisionTreeClassifier(min_samples_leaf=20, random_state=42).fit(X_train, y_train)\nimportances = pd.Series(leafy.feature_importances_, index=features)\ntop_feature = importances.idxmax()\nprint(top_feature)\n","checks":[{"id":"rules-test","test":"list(map(int, rule_preds)) == tree.predict(X_test).tolist()"},{"id":"rules-train","test":"[int(rule_predict(r)) for _, r in X_train.iterrows()] == tree.predict(X_train).tolist()"},{"id":"leafy","test":"leafy is not None and hasattr(leafy, 'tree_') and leafy.min_samples_leaf == 20 and leafy.get_n_leaves() == __import__('sklearn.tree', fromlist=['x']).DecisionTreeClassifier(min_samples_leaf=20, random_state=42).fit(X_train, y_train).get_n_leaves()"},{"id":"top-feature","test":"top_feature == 'days_since_last_order'"}],"title":"Turn a tree into rules","prompt":"A tree is a stack of if/else statements. Prove it.
\nrule_predict(row) so it returns exactly what the tree predicts (1 = churn, 0 = stay) for any customer row. The checks compare it with tree.predict on every training and test row.leafy: a DecisionTreeClassifier(min_samples_leaf=20, random_state=42) fitted on the training data.top_feature: the name of the feature with the largest feature_importances_ value in leafy.|--- line is a condition; the class: lines are the return values.","Under days_since_last_order <= 236.5 the tree asks about orders; under the other branch it asks about tenure_days.","pd.Series(leafy.feature_importances_, index=features).idxmax() returns the top feature’s name."],"explanation":"Four leaves become four return statements. The recent-customer branch depends on order count, and the long-silent branch is almost always churn except for the tiny group of three very long-tenured customers. That’s the interaction the tree found on its own. Code like this could run in a SQL CASE expression, which is one reason shallow trees are easy to deploy. days_since_last_order dominates the importances of the bigger tree too.