Models in scikit-learn: fit, predict, score

scikit-learn ships dozens of models, from straight lines to forests of trees, and you use every one of them the same way. Learn three method calls and you can try any of them on the churn data in a few lines. That consistency is the main reason scikit-learn is the default library for classic machine learning.

Three calls

A model in scikit-learn is an estimator: a Python object you create with its settings, train with fit, and use with predict and score.

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier

churn = pd.read_csv("churn.csv")
features = ["tenure_days", "orders", "total_spent", "avg_order_value",
            "days_since_last_order", "used_coupon", "support_tickets", "mobile_share"]
X, y = churn[features], churn["churned"]
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y)

model = DecisionTreeClassifier(max_depth=3, random_state=42)  # 1. create
model.fit(X_train, y_train)                                    # 2. learn
print(model.predict(X_test)[:10])                              # 3. use
print("train accuracy:", round(model.score(X_train, y_train), 3))
print("test accuracy: ", round(model.score(X_test, y_test), 3))

What each call does:

• The constructor takes hyperparameters: choices you make before training. max_depth=3 says the tree may ask at most three questions in a row. random_state=42 makes tie-breaking repeatable.
• fit(X_train, y_train) looks at the examples and learns the model’s parameters. For a tree, those are the questions and thresholds (“is days_since_last_order above 236.5?”). It returns the model itself, so model = DecisionTreeClassifier().fit(X, y) also works.
• predict(X) returns one prediction per row: here an array of 0s and 1s.
• score(X, y) predicts and compares with the true answers. For classifiers it returns accuracy, the share of correct predictions. For regressors it returns R², which you’ll meet in section 2.

What a fitted model knows

Everything a model learns during fit is stored in attributes whose names end with an underscore. They don’t exist before fit; asking for one raises an AttributeError, and calling predict on an unfitted model raises NotFittedError.

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier

churn = pd.read_csv("churn.csv")
features = ["tenure_days", "orders", "total_spent", "avg_order_value",
            "days_since_last_order", "used_coupon", "support_tickets", "mobile_share"]
X_train, X_test, y_train, y_test = train_test_split(
    churn[features], churn["churned"], test_size=0.25, random_state=42,
    stratify=churn["churned"])
model = DecisionTreeClassifier(max_depth=3, random_state=42).fit(X_train, y_train)

print(model.classes_)            # the labels it saw: [0 1]
print(model.n_features_in_)      # 8
print(model.feature_names_in_[:3])

new_customer = pd.DataFrame([{
    "tenure_days": 400, "orders": 2, "total_spent": 180.0, "avg_order_value": 90.0,
    "days_since_last_order": 150, "used_coupon": 0, "support_tickets": 1,
    "mobile_share": 0.5}])
print(model.predict(new_customer))        # [1]: likely to churn
print(model.predict_proba(new_customer))  # [[0.36 0.64]]

predict_proba gives the model’s estimated probability for each class, in the order of classes_. This customer gets 64% for churn. You’ll lean on probabilities heavily in section 3, because they let you choose how cautious to be.

The training score lies

Now remove the depth limit and watch what happens. An unlimited tree keeps asking questions until every training customer sits in a leaf with only their own class.

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.neighbors import KNeighborsClassifier

churn = pd.read_csv("churn.csv")
features = ["tenure_days", "orders", "total_spent", "avg_order_value",
            "days_since_last_order", "used_coupon", "support_tickets", "mobile_share"]
X_train, X_test, y_train, y_test = train_test_split(
    churn[features], churn["churned"], test_size=0.25, random_state=42,
    stratify=churn["churned"])

for model in [DecisionTreeClassifier(max_depth=3, random_state=42),
              DecisionTreeClassifier(random_state=42),
              KNeighborsClassifier(n_neighbors=5)]:
    model.fit(X_train, y_train)
    print(f"{model!r:55} train {model.score(X_train, y_train):.2f}  test {model.score(X_test, y_test):.2f}")

The unlimited tree is perfect on the 298 customers it studied and worse than the shallow tree on the 100 it didn’t. It memorised quirks of individual customers. This gap between training and test performance is overfitting, and you’ll spend much of section 2 learning to control it.

Notice also the third model. k-nearest neighbours works nothing like a tree (it predicts by looking at the five most similar training customers), yet the code is identical. Swapping models costs one line.

Same contract, different assumptions

The shared API hides very different ideas about the world. A tree assumes churn can be described by a handful of yes/no questions with sharp cut-offs. k-nearest neighbours assumes customers who look alike behave alike, where “look alike” means close together when every column is treated as a distance. Neither assumption is right; each is useful in some situations and wrong in others. That’s why you try several models on the same split rather than arguing about which one ought to win.

Regression models keep the same contract. DecisionTreeRegressor or LinearRegression take a numeric y, predict returns numbers instead of classes, and score returns R² instead of accuracy. Everything you learn about one estimator carries over to the rest.

Two details save confusion later. Calling fit a second time throws away what the model learned before and starts from scratch; it doesn’t add to the old knowledge. And the settings you passed to the constructor stay readable as plain attributes (model.max_depth), so you can always check what a fitted model was told to do.

You now have a model at 73% test accuracy. Is that good? You can’t say yet, because you haven’t asked what a model with no intelligence at all would score. That’s the next lesson.