Shipping a Model

A model in a notebook helps nobody. To change what Cartwheel does, the champion pipeline has to run on fresh customer data, on a schedule or on request, and its scores have to reach the people who send the win-back offers. This lesson covers the last mile: saving the model, guarding its inputs, choosing between a batch job and an API, and proving it works.

Save the pipeline, not just the model

The thing to ship is the whole fitted pipeline. The scaler’s means and standard deviations are as much a part of the model as the coefficients; without them, production would scale new customers differently and every score would be off.

scikit-learn’s model persistence guide covers several formats (pickle, joblib, skops, ONNX); joblib is the most common when Python both saves and loads the model. Save a small bundle with what someone will need later: the pipeline, the exact feature list, a version label and the library version it was trained with.

import os
import tempfile

import joblib
import pandas as pd
import sklearn
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

churn = pd.read_csv("churn.csv")
features = ["tenure_days", "orders", "total_spent", "avg_order_value",
            "days_since_last_order", "used_coupon", "support_tickets", "mobile_share"]
champion = make_pipeline(StandardScaler(), LogisticRegression()).fit(churn[features], churn["churned"])

bundle = {"model": champion, "features": features,
          "version": "churn-2025-09-30.1", "sklearn_version": sklearn.__version__}

with tempfile.TemporaryDirectory() as folder:
    path = os.path.join(folder, "churn_model.joblib")
    joblib.dump(bundle, path)
    print("saved", os.path.getsize(path), "bytes")

    loaded = joblib.load(path)
    same = (loaded["model"].predict_proba(churn[features]) == champion.predict_proba(churn[features])).all()
    print("identical predictions after reload:", same, "| trained with scikit-learn", loaded["sklearn_version"])

Notice the final model is fitted on all 398 labelled customers. The test set’s job was to measure the modelling process; once you’ve measured it, retraining on every labelled row gives the deployed model a little more to learn from. Record that you did it, and keep the reported metric from the test-set evaluation.

Guard the inputs

A model will happily score nonsense. A customer with orders = 0, a mobile_share of 7, or a missing column produces a probability that looks as confident as any other. Put a thin layer in front of predict_proba that checks the input and fails loudly:

# assumes `import pandas as pd` and a loaded `bundle` as above
def predict_churn(record: dict, bundle: dict) -> float:
    missing = [f for f in bundle["features"] if f not in record]
    if missing:
        raise ValueError(f"missing fields: {missing}")
    if not 0 <= record["mobile_share"] <= 1:
        raise ValueError("mobile_share must be between 0 and 1")
    row = pd.DataFrame([record])[bundle["features"]]
    return float(bundle["model"].predict_proba(row)[0, 1])

Selecting [bundle["features"]] fixes the column order, which, as section 1 showed, scikit-learn insists on.

Batch job or API?

There are two common ways to put a model to work:

• Batch scoring: a scheduled job (nightly or weekly) loads the model, scores every active customer from the warehouse and writes the probabilities to a table. Marketing’s tools read the table. Simple, cheap, easy to monitor, and right for churn, where a decision a day later costs nothing.
• Online API: a web service scores one customer per request, in milliseconds. Needed when the decision happens during a user’s session, such as showing a retention offer on the cancellation page.

If you do need an API, a small FastAPI service is a common choice in Python. This sketch isn’t runnable in the browser, but it’s complete enough to run locally with uvicorn app:app:

import joblib
import pandas as pd
from fastapi import FastAPI
from pydantic import BaseModel, Field

# joblib uses pickle: load only model files from your own trusted, access-controlled store
bundle = joblib.load("churn_model.joblib")
app = FastAPI()

class Customer(BaseModel):
    tenure_days: int = Field(ge=0)
    orders: int = Field(ge=1)
    total_spent: float = Field(ge=0)
    avg_order_value: float = Field(ge=0)
    days_since_last_order: int = Field(ge=0)
    used_coupon: int = Field(ge=0, le=1)
    support_tickets: int = Field(ge=0)
    mobile_share: float = Field(ge=0, le=1)

@app.post("/predict")
def predict(customer: Customer):
    row = pd.DataFrame([customer.model_dump()])[bundle["features"]]
    prob = float(bundle["model"].predict_proba(row)[0, 1])
    return {"churn_probability": round(prob, 3), "model_version": bundle["version"]}

The Pydantic model does the input validation declaratively: a request with orders: 0 or a missing field gets a 422 error before the model ever sees it. Returning the model version with every answer makes it possible to trace any prediction back to the exact model that made it. Log every request and response; those logs are what the drift checks from the previous lesson run on.

Did it work? Run an experiment

A good ROC AUC means the model ranks customers well. It doesn’t mean the win-back offer changes anyone’s mind. To measure that, hold out a random slice of flagged customers (say 20%) who don’t get the offer. After 90 days, compare churn between the contacted and held-out groups. The difference is the effect of the offer on the customers the model picks, and multiplied by customer value it’s the number the business actually cares about. Without the held-out group, any drop in churn could be the season, a competitor’s stumble or luck.

Where you are now

You started with a question and a table, and you now have a process that holds up: frame the problem, split before you look, beat a majority-class dummy and a rule, choose metrics by the cost of each mistake, compare models with cross-validation inside leak-free pipelines, check who the model serves worse, and ship something you can monitor and test. The champion turned out to be the simplest serious model, and you know exactly why. That habit of asking “compared to what?” is the most valuable thing this course had to teach, and it applies unchanged to the next model you build, whatever it is.