Pipelines and ColumnTransformer

Section 3 kept leaving data on the table. country and segment are text, which no scikit-learn model accepts directly. avg_satisfaction has gaps that logistic regression refuses. And you’ve been scaling by hand, keeping track of X_train_s and X_test_s. Each of those steps has to be learned from training data only, applied identically to new data, and redone inside every cross-validation fold. Doing that by hand is how leaks and bugs happen. A pipeline does it for you.

Pipelines bundle steps into one estimator

A Pipeline is a list of steps: any number of transformers (objects with fit and transform, such as StandardScaler) followed by one final estimator. The pipeline behaves like a single model:

• pipe.fit(X, y) fits the first transformer, transforms the data, passes it to the next step, and so on, then fits the model on the result.
• pipe.predict(X_new) runs X_new through the already-fitted transformers, then predicts.

You’ve used make_pipeline, which names steps automatically ("standardscaler", "logisticregression"). Pipeline([("prep", ...), ("model", ...)]) lets you choose the names, which you’ll want next lesson when you tune parameters by name.

The real payoff is in cross-validation. cross_validate clones the entire pipeline and refits it in every fold, so the scaler’s means, the imputer’s medians and the encoder’s category list are learned from that fold’s training rows only.

Different columns, different treatment

Not every column needs the same preprocessing. ColumnTransformer routes each group of columns through its own transformer and glues the outputs side by side.

Three transformers do the work:

• OneHotEncoder creates one 0/1 column per category: country_Egypt, country_Germany and so on. Set handle_unknown="ignore"; the default raises an error the first time a customer arrives from a country the training data didn’t include, which in production means a crashed prediction service.
• SimpleImputer fills missing values with the mean, median, most frequent value or a constant, learned from training data. With add_indicator=True it also adds a column that is 1 where the value was missing. Half of Cartwheel’s customers never answered the survey, and that fact may matter more than the score itself.
• StandardScaler, as before. A small make_pipeline lets one column group pass through two steps in order.

import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold, cross_val_score, train_test_split
from sklearn.pipeline import Pipeline, make_pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

churn = pd.read_csv("churn.csv")
X = churn.drop(columns=["customer_id", "churned"])
y = churn["churned"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42, stratify=y)

numeric = ["tenure_days", "orders", "total_spent", "avg_order_value",
           "days_since_last_order", "used_coupon", "support_tickets", "mobile_share"]
preprocess = ColumnTransformer([
    ("num", StandardScaler(), numeric),
    ("sat", make_pipeline(SimpleImputer(strategy="median", add_indicator=True), StandardScaler()),
     ["avg_satisfaction"]),
    ("cat", OneHotEncoder(handle_unknown="ignore"), ["country", "segment"]),
])
pipe = Pipeline([("prep", preprocess), ("model", LogisticRegression())])

pipe.fit(X_train, y_train)
names = pipe.named_steps["prep"].get_feature_names_out()
print(len(names), "model inputs, e.g.", list(names[8:12]))

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
print("all columns  ROC AUC:", cross_val_score(pipe, X_train, y_train, cv=cv, scoring="roc_auc").mean().round(3))
numeric_only = make_pipeline(StandardScaler(), LogisticRegression())
print("numeric only ROC AUC:", cross_val_score(numeric_only, X_train[numeric], y_train, cv=cv, scoring="roc_auc").mean().round(3))

The pipeline accepts the raw DataFrame, text and gaps included, and produces 21 model inputs: eight scaled numbers, the imputed satisfaction score and its missing flag, eight country columns and three segment columns. get_feature_names_out lists them with a prefix naming the transformer that made each one, which is how you match coefficients to columns later.

More columns, worse model

Then the surprise: the full pipeline ranks customers worse than the eight numeric columns alone, 0.776 against 0.807. Eleven one-hot columns and a satisfaction score with half its values imputed give logistic regression more coefficients to fit with only 298 rows, and on this data the extra columns carry more noise than signal. Egypt’s churn rate does look higher in the training data, but with 44 Egyptian customers in training that’s too thin to bet on.

That’s not a failure of the pipeline. It’s the pipeline doing its job: it made testing the extra columns a five-line change, and cross-validation gave a clear answer. For now, the numeric model stays the champion. The categorical columns are worth revisiting when Cartwheel has a few thousand customers, or with stronger regularisation.

A pipeline also gives every setting an address: model__C is the C of the step named model, and prep__sat__simpleimputer__strategy reaches the imputer. Next lesson uses those addresses to search for the best settings automatically.