{"id":"l-ml-3-1","courseId":"ml-crash-course","lang":"en","kind":"lesson","summary":"Train a logistic regression on the churn data, read its probabilities and coefficients, and use the probabilities to rank customers so marketing contacts the riskiest first.","html":"

The depth-3 tree from section 1 answered “churn” or “stay”. Marketing doesn’t really want that. They have budget for a few dozen win-back calls a week and want to know who to call first. That needs a score that ranks customers by risk, and the classic model for it is logistic regression.

\n

From a line to a probability

\n

Logistic regression starts like linear regression: multiply each feature by a coefficient, add them up, add an intercept. That gives a score z that can be any number. Then it squashes the score through the sigmoid function, 1 / (1 + e^-z), which maps any number to a value between 0 and 1. Large positive scores give probabilities near 1, large negative scores near 0, and a score of exactly 0 gives 0.5.

\n
\n An S-shaped curve. The horizontal axis is the linear score z from minus 6 to plus 6; the vertical axis is probability from 0 to 1. The curve passes through 0.5 at z equals 0 and flattens toward 0 and 1 at the extremes.\n \n \n \n \n 1.0\n 0.5\n 0.0\n z = −6\n 0\n +6\n \n \n likely stays\n likely churns\n
The sigmoid turns a score into a probability
\n

During training, the model adjusts the coefficients so that churners get high probabilities and non-churners low ones. The loss it minimises is log loss, which punishes confident wrong answers hard: predicting 0.99 for a customer who stayed costs far more than predicting 0.6. There’s no exact formula for the minimum, so scikit-learn finds it with an iterative optimiser, a refined cousin of the gradient descent you wrote last section.

\n

Fitting it on churn

\n

Because the optimiser is iterative and the model has a penalty (more on that below), scale the features first. A pipeline keeps the scaler and the model together.

\n
Python
import pandas as pd\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.preprocessing import StandardScaler\n\nchurn = pd.read_csv(\"churn.csv\")\nfeatures = [\"tenure_days\", \"orders\", \"total_spent\", \"avg_order_value\",\n            \"days_since_last_order\", \"used_coupon\", \"support_tickets\", \"mobile_share\"]\nX_train, X_test, y_train, y_test = train_test_split(\n    churn[features], churn[\"churned\"], test_size=0.25, random_state=42,\n    stratify=churn[\"churned\"])\n\nmodel = make_pipeline(StandardScaler(), LogisticRegression()).fit(X_train, y_train)\nprint(\"test accuracy:\", model.score(X_test, y_test))\nprint(model.predict_proba(X_test)[:4].round(2))\nprint(model.predict(X_test)[:4])
\n

predict_proba returns two columns, one per class in the order of classes_: probability of staying, probability of churning. They add up to 1. predict is a shortcut that labels a row 1 when the churn probability is at least 0.5. That 0.5 is a default, not a law, and section 3’s last lesson is about choosing it on purpose.

\n

Accuracy is 0.72, level with the 80-day rule. Don’t stop reading there: the probabilities carry information accuracy throws away.

\n

Reading the coefficients

\n

With standardised features, each coefficient says how much one standard deviation of that feature moves the score z (the log-odds of churn). That makes them comparable across features.

\n
Python
import pandas as pd\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.preprocessing import StandardScaler\n\nchurn = pd.read_csv(\"churn.csv\")\nfeatures = [\"tenure_days\", \"orders\", \"total_spent\", \"avg_order_value\",\n            \"days_since_last_order\", \"used_coupon\", \"support_tickets\", \"mobile_share\"]\nX_train, X_test, y_train, y_test = train_test_split(\n    churn[features], churn[\"churned\"], test_size=0.25, random_state=42,\n    stratify=churn[\"churned\"])\nmodel = make_pipeline(StandardScaler(), LogisticRegression()).fit(X_train, y_train)\n\ncoefs = pd.Series(model[-1].coef_[0], index=features).sort_values()\nprint(coefs.round(2))\n\nranked = X_test.assign(churn_prob=model.predict_proba(X_test)[:, 1], churned=y_test)\ntop10 = ranked.sort_values(\"churn_prob\", ascending=False).head(10)\nprint(top10[[\"days_since_last_order\", \"orders\", \"churn_prob\", \"churned\"]].round(2))\nprint(\"share of the top 10 who really churned:\", top10[\"churned\"].mean())
\n

Recency (days_since_last_order, +0.93) dominates, which matches what the rule baseline told you. Longer tenure also pushes towards churn, while more orders, more spend, coupon use and even more support tickets push away from it. Customers who contact support are engaged customers; it’s the silent ones who leave. That’s the kind of finding worth taking back to the business, with the caveat that it’s an association, not a cause.

\n

Then the payoff: of the ten customers the model ranks riskiest, nine really churned. A yes/no rule can’t produce that list; a probability can.

\n\n

The penalty you didn’t ask for

\n

LogisticRegression is regularised by default with an L2 penalty, the same idea as Ridge. Its strength is set by C, which is the inverse of Ridge’s alpha: small C means a strong penalty, large C means a weak one. The default C=1.0 is a reasonable start. On this data, values from 0.01 to 100 all land within a couple of points of each other, a sign that eight well-behaved features don’t give the model much room to overfit. With hundreds of features, C matters a lot, and you’d tune it with the tools in section 4.

\n\n

Logistic regression draws one straight boundary through the feature space. Next lesson’s model makes no such assumption: it asks a series of yes/no questions and can carve the space into boxes.

\n","toc":[{"id":"sec-1","text":"From a line to a probability","level":2},{"id":"sec-2","text":"Fitting it on churn","level":2},{"id":"sec-3","text":"Reading the coefficients","level":2},{"id":"sec-4","text":"The penalty you didn’t ask for","level":2}],"takeaways":["Logistic regression computes a linear score and squashes it through the sigmoid into a probability between 0 and 1.","predict_proba returns one probability per class; predict applies a 0.5 cut-off to it.","On standardised features, each coefficient is the change in log-odds per standard deviation, so its sign and size are comparable across features.","LogisticRegression is regularised by default; C is the inverse of the penalty strength, so a smaller C means a stronger penalty.","Ranking customers by probability is often more useful than a yes/no label, because it lets the business choose how many to act on."],"further":[{"title":"Logistic regression","url":"https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression"},{"title":"LogisticRegression","url":"https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html"}],"quiz":[{"q":"

A customer’s linear score inside a logistic regression is 0. What churn probability does the model give?

","options":[{"text":"0.5, because the sigmoid of 0 is exactly one half.","why":"Correct. A score of 0 means the evidence for and against churn balances."},{"text":"0, because a score of 0 means no churn.","why":"The probability only approaches 0 as the score goes far negative. Zero sits in the middle of the curve."},{"text":"It depends on the intercept, which is added after the sigmoid.","why":"The intercept is part of the linear score, added before the sigmoid. Once the score is 0, the output is fixed."},{"text":"1, because the sigmoid of 0 is 1.","why":"The sigmoid is 1 / (1 + e^-z); with z = 0 that’s 1 / 2."}],"answer":0},{"q":"

On standardised churn features, days_since_last_order has coefficient +0.93 and total_spent has -0.26. What can you say?

","options":[{"text":"Each extra day of recency matters about 3.6 times as much as each extra dollar of spend.","why":"The coefficients are per standard deviation, not per dollar or per day, so a per-unit comparison doesn’t follow."},{"text":"Customers who spent more churn more often.","why":"A negative coefficient means more spending is associated with lower churn odds, holding the other features fixed."},{"text":"Raising spend by marketing would cut churn by a known amount.","why":"Coefficients describe associations in this data; they don’t tell you the effect of an intervention."},{"text":"A one-standard-deviation increase in recency raises churn odds far more than the same increase in spend lowers them.","why":"Correct. On a shared scale, the size of the coefficient compares effects in the fitted model."}],"answer":3},{"q":"

You fit LogisticRegression() on the raw churn features and get a ConvergenceWarning. What is the best first fix?

","options":[{"text":"Ignore it, because the accuracy looks fine.","why":"The optimiser stopped before finishing, so the coefficients are not the ones the model defines. Fix the cause."},{"text":"Lower C to 0.0001 so the model has less to learn.","why":"A tiny C hides the symptom by flattening the model, at the cost of a very different, underfit model."},{"text":"Add a StandardScaler before the model in a pipeline.","why":"Correct. Features on wildly different scales make the optimiser’s job hard; scaling usually removes the warning."}],"answer":2},{"q":"

Marketing can call 30 customers this week. What should your model give them?

","options":[{"text":"The 30 customers with the highest predicted churn probability.","why":"Correct. Ranking by probability puts the limited budget where the model is most confident churn is coming."},{"text":"Any 30 customers that predict labels as 1.","why":"Dozens of customers may be labelled 1. Picking 30 at random among them wastes the ranking the model already computed."},{"text":"The 30 customers whose probability is closest to 0.5.","why":"Those are the cases the model is least sure about, not the ones most likely to churn."}],"answer":0}],"exercise":{"kind":"python","files":["churn.csv"],"starter":"import pandas as pd\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.preprocessing import StandardScaler\n\nchurn = pd.read_csv(\"churn.csv\")\nfeatures = [\"tenure_days\", \"orders\", \"total_spent\", \"avg_order_value\",\n \"days_since_last_order\", \"used_coupon\", \"support_tickets\", \"mobile_share\"]\nX_train, X_test, y_train, y_test = train_test_split(\n churn[features], churn[\"churned\"], test_size=0.25, random_state=42,\n stratify=churn[\"churned\"])\n\n# 1. A pipeline: StandardScaler, then LogisticRegression; fit it on the training data\nmodel = LogisticRegression().fit(X_train, y_train)\n\n# 2. Churn probability (class 1) for every test customer\nchurn_prob = model.predict(X_test)\n\n# 3. customer_id of the 20 test customers with the highest churn probability\ncall_list = []\n\n# 4. Share of those 20 who really churned\nhit_rate = None\n","solution":"import pandas as pd\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.preprocessing import StandardScaler\n\nchurn = pd.read_csv(\"churn.csv\")\nfeatures = [\"tenure_days\", \"orders\", \"total_spent\", \"avg_order_value\",\n \"days_since_last_order\", \"used_coupon\", \"support_tickets\", \"mobile_share\"]\nX_train, X_test, y_train, y_test = train_test_split(\n churn[features], churn[\"churned\"], test_size=0.25, random_state=42,\n stratify=churn[\"churned\"])\n\nmodel = make_pipeline(StandardScaler(), LogisticRegression()).fit(X_train, y_train)\n\nchurn_prob = model.predict_proba(X_test)[:, 1]\n\nranked = churn.loc[X_test.index, [\"customer_id\", \"churned\"]].assign(prob=churn_prob)\ntop20 = ranked.sort_values(\"prob\", ascending=False).head(20)\ncall_list = top20[\"customer_id\"].tolist()\n\nhit_rate = top20[\"churned\"].mean()\nprint(call_list[:5], hit_rate)\n","checks":[{"id":"pipeline","test":"[type(s).__name__ for s in model] == ['StandardScaler', 'LogisticRegression']"},{"id":"probs","test":"len(churn_prob) == 100 and abs(float(churn_prob[0]) - float(__import__('sklearn.pipeline', fromlist=['x']).make_pipeline(__import__('sklearn.preprocessing', fromlist=['x']).StandardScaler(), __import__('sklearn.linear_model', fromlist=['x']).LogisticRegression()).fit(X_train, y_train).predict_proba(X_test)[:, 1][0])) < 1e-6 and 0 < float(min(churn_prob)) < float(max(churn_prob)) < 1"},{"id":"call-list","test":"len(call_list) == 20 and set(int(c) for c in call_list) == set(churn.loc[X_test.index, 'customer_id'].to_numpy()[__import__('sklearn.pipeline', fromlist=['x']).make_pipeline(__import__('sklearn.preprocessing', fromlist=['x']).StandardScaler(), __import__('sklearn.linear_model', fromlist=['x']).LogisticRegression()).fit(X_train, y_train).predict_proba(X_test)[:, 1].argsort()[::-1][:20]].tolist())"},{"id":"hit-rate","test":"hit_rate is not None and abs(float(hit_rate) - churn.set_index('customer_id').loc[[int(c) for c in call_list], 'churned'].mean()) < 1e-9 and float(hit_rate) > 0.8"}],"title":"Build Monday's call list","prompt":"

Marketing can call 20 customers. Give them the right 20.

\n
    \n
  1. model: a pipeline of StandardScaler then LogisticRegression, fitted on the training data.
    2. \n
  2. churn_prob: the churn probability (class 1) for each test customer.
    3. \n
  3. call_list: the customer_id values of the 20 test customers with the highest churn probability.
    4. \n
  4. hit_rate: the share of those 20 who actually churned.
    5. \n
\n","hints":["make_pipeline(StandardScaler(), LogisticRegression())","predict gives labels. predict_proba(X_test)[:, 1] gives the churn column of probabilities.","X_test keeps the original row index, so churn.loc[X_test.index, "customer_id"] lines up with churn_prob. Add the probabilities as a column, sort descending, take head(20)."],"explanation":"

Ranking by probability concentrates the call budget: well over 80% of the 20 riskiest test customers really churned, against 55% for a random pick. predict would only have said “1” for dozens of customers without any order among them. Keeping the original index from train_test_split is what lets you join predictions back to ids.

\n","checkLabels":{"pipeline":"model is StandardScaler → LogisticRegression","probs":"churn_prob holds the class-1 probabilities","call-list":"call_list has the 20 highest-risk customer ids","hit-rate":"hit_rate is the share of the call list who churned"}},"words":690,"readMinutes":3}