Clustering and PCA

Not every question comes with labels. Cartwheel’s marketing team asks: “What kinds of customers do we have?” Nobody has tagged customers with types, so there’s no target to predict. Unsupervised learning looks for structure in the features alone. The two tools you’ll use most are clustering, which groups similar rows, and dimensionality reduction, which summarises many columns with a few.

k-means in two moves

k-means needs one decision from you: the number of clusters, k. Then it repeats two steps:

1. Assign: put each customer in the cluster whose centre is nearest.
2. Update: move each centre to the average of the customers assigned to it.

When assignments stop changing, it’s done. Different starting centres can give different results, so scikit-learn runs the whole thing n_init times and keeps the tightest solution.

Because “nearest” is a distance, the same lesson from k-nearest neighbours applies: scale first, or total_spent in thousands decides everything.

Segmenting Cartwheel’s customers

Use behaviour columns only and leave churned out: a segmentation should describe customers, not their outcome. You’ll look at churn afterwards to see whether the segments mean anything.

import pandas as pd
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

churn = pd.read_csv("churn.csv")
behaviour = ["orders", "total_spent", "avg_order_value", "days_since_last_order",
             "tenure_days", "mobile_share"]
X = StandardScaler().fit_transform(churn[behaviour])
for k in range(2, 7):
    labels = KMeans(n_clusters=k, n_init=10, random_state=42).fit_predict(X)
    print(f"k={k}  silhouette {silhouette_score(X, labels):.3f}")

segments = make_pipeline(StandardScaler(), KMeans(n_clusters=4, n_init=10, random_state=42))
churn["segment_id"] = segments.fit_predict(churn[behaviour])
profile = churn.groupby("segment_id").agg(
    customers=("orders", "size"), orders=("orders", "mean"),
    avg_order=("avg_order_value", "mean"), days_since=("days_since_last_order", "mean"),
    tenure=("tenure_days", "mean"), churn_rate=("churned", "mean"))
print(profile.round(2))

First, the honest part. The silhouette score measures how much closer each customer is to its own cluster than to the next one, from -1 to 1. Scores around 0.25 for every k mean the customers don’t fall into crisp natural groups; they form a continuum. That’s typical for behavioural data. It doesn’t make clustering useless, but it means k is your choice, made for usefulness, not a fact you discovered.

Then the useful part. Profiling the four clusters gives segments a marketer can name:

• Lapsed occasionals (129): two orders, last one about 310 days ago. 79% churn.
• Big-ticket buyers (41): few orders averaging $600. 66% churn.
• Regulars (79): eight orders, the last about 100 days ago. 33% churn.
• New and recent (149): short tenure, ordered about two months ago. 44% churn.

Churn rates differ sharply between segments even though k-means never saw churned. That’s evidence the segments capture something real, and it gives marketing four different conversations to have instead of one.

k-means isn’t the only option. It assumes roughly round clusters of similar size, an assumption that is usually good enough for behavioural segments. Hierarchical clustering (AgglomerativeClustering) shows how groups merge at different levels, and DBSCAN finds dense regions of any shape and labels outliers as noise. Start with k-means for segmentation; switch when its assumptions clearly don’t fit your data.

PCA: fewer columns, most of the information

Many datasets have dozens of correlated columns. The breast cancer data has 30 measurements of cell nuclei, many of them variations on size. Principal component analysis finds new axes, each a weighted mix of the original columns, ordered by how much of the data’s variation they capture. Keep the first few and you have a compact summary.

import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.decomposition import PCA
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

X, y = load_breast_cancer(return_X_y=True)
pca = make_pipeline(StandardScaler(), PCA()).fit(X)
ratios = pca[-1].explained_variance_ratio_
print("first two components keep:", ratios[:2].round(3), "=", round(ratios[:2].sum(), 3))
print("components needed for 90%:", int(np.argmax(np.cumsum(ratios) >= 0.90)) + 1)

coords = make_pipeline(StandardScaler(), PCA(n_components=2)).fit_transform(X)
for label, name in [(0, "malignant"), (1, "benign")]:
    print(name, "average position:", coords[y == label].mean(axis=0).round(2))

Two components keep 63% of the variation in 30 scaled columns, and seven keep 90%. Plot the two coordinates and malignant and benign tumours sit on opposite sides of the first axis, even though PCA never saw the labels. That’s why a 2D PCA plot is a standard first look at any wide dataset.

Both techniques here are classic and cheap. The next lesson is about models that learn their own features from raw inputs: neural networks.