{"id":"l-ml-5-5","courseId":"ml-crash-course","lang":"en","kind":"lesson","summary":"Save the champion pipeline with joblib, wrap it in a validated prediction function, serve it as a nightly batch job or an API, and measure whether it changes business outcomes.","html":"

A model in a notebook helps nobody. To change what Cartwheel does, the champion pipeline has to run on fresh customer data, on a schedule or on request, and its scores have to reach the people who send the win-back offers. This lesson covers the last mile: saving the model, guarding its inputs, choosing between a batch job and an API, and proving it works.

\n

Save the pipeline, not just the model

\n

The thing to ship is the whole fitted pipeline. The scaler’s means and standard deviations are as much a part of the model as the coefficients; without them, production would scale new customers differently and every score would be off.

\n

scikit-learn’s model persistence guide covers several formats (pickle, joblib, skops, ONNX); joblib is the most common when Python both saves and loads the model. Save a small bundle with what someone will need later: the pipeline, the exact feature list, a version label and the library version it was trained with.

\n
Python
import os\nimport tempfile\n\nimport joblib\nimport pandas as pd\nimport sklearn\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.preprocessing import StandardScaler\n\nchurn = pd.read_csv(\"churn.csv\")\nfeatures = [\"tenure_days\", \"orders\", \"total_spent\", \"avg_order_value\",\n            \"days_since_last_order\", \"used_coupon\", \"support_tickets\", \"mobile_share\"]\nchampion = make_pipeline(StandardScaler(), LogisticRegression()).fit(churn[features], churn[\"churned\"])\n\nbundle = {\"model\": champion, \"features\": features,\n          \"version\": \"churn-2025-09-30.1\", \"sklearn_version\": sklearn.__version__}\n\nwith tempfile.TemporaryDirectory() as folder:\n    path = os.path.join(folder, \"churn_model.joblib\")\n    joblib.dump(bundle, path)\n    print(\"saved\", os.path.getsize(path), \"bytes\")\n\n    loaded = joblib.load(path)\n    same = (loaded[\"model\"].predict_proba(churn[features]) == champion.predict_proba(churn[features])).all()\n    print(\"identical predictions after reload:\", same, \"| trained with scikit-learn\", loaded[\"sklearn_version\"])
\n

Notice the final model is fitted on all 398 labelled customers. The test set’s job was to measure the modelling process; once you’ve measured it, retraining on every labelled row gives the deployed model a little more to learn from. Record that you did it, and keep the reported metric from the test-set evaluation.

\n\n

Guard the inputs

\n

A model will happily score nonsense. A customer with orders = 0, a mobile_share of 7, or a missing column produces a probability that looks as confident as any other. Put a thin layer in front of predict_proba that checks the input and fails loudly:

\n
Python
# assumes `import pandas as pd` and a loaded `bundle` as above\ndef predict_churn(record: dict, bundle: dict) -> float:\n    missing = [f for f in bundle[\"features\"] if f not in record]\n    if missing:\n        raise ValueError(f\"missing fields: {missing}\")\n    if not 0 <= record[\"mobile_share\"] <= 1:\n        raise ValueError(\"mobile_share must be between 0 and 1\")\n    row = pd.DataFrame([record])[bundle[\"features\"]]\n    return float(bundle[\"model\"].predict_proba(row)[0, 1])
\n

Selecting [bundle["features"]] fixes the column order, which, as section 1 showed, scikit-learn insists on.

\n

Batch job or API?

\n

There are two common ways to put a model to work:

\n\n
\n A training pipeline produces a versioned model file. The model file feeds either a nightly batch job that writes scores to a table for marketing, or an online API that answers single requests. Both log inputs and predictions to monitoring, which feeds back into retraining.\n \n training\n pipeline + CV\n \n \n model.joblib\n versioned\n \n \n \n nightly batch job\n scores → table\n \n online API\n one request, ms\n \n \n \n monitoring\n logs, drift\n \n retrain when monitoring says so\n
From training to decisions
\n

If you do need an API, a small FastAPI service is a common choice in Python. This sketch isn’t runnable in the browser, but it’s complete enough to run locally with uvicorn app:app:

\n
app.py
import joblib\nimport pandas as pd\nfrom fastapi import FastAPI\nfrom pydantic import BaseModel, Field\n\n# joblib uses pickle: load only model files from your own trusted, access-controlled store\nbundle = joblib.load(\"churn_model.joblib\")\napp = FastAPI()\n\nclass Customer(BaseModel):\n    tenure_days: int = Field(ge=0)\n    orders: int = Field(ge=1)\n    total_spent: float = Field(ge=0)\n    avg_order_value: float = Field(ge=0)\n    days_since_last_order: int = Field(ge=0)\n    used_coupon: int = Field(ge=0, le=1)\n    support_tickets: int = Field(ge=0)\n    mobile_share: float = Field(ge=0, le=1)\n\n@app.post(\"/predict\")\ndef predict(customer: Customer):\n    row = pd.DataFrame([customer.model_dump()])[bundle[\"features\"]]\n    prob = float(bundle[\"model\"].predict_proba(row)[0, 1])\n    return {\"churn_probability\": round(prob, 3), \"model_version\": bundle[\"version\"]}
\n

The Pydantic model does the input validation declaratively: a request with orders: 0 or a missing field gets a 422 error before the model ever sees it. Returning the model version with every answer makes it possible to trace any prediction back to the exact model that made it. Log every request and response; those logs are what the drift checks from the previous lesson run on.

\n

Did it work? Run an experiment

\n

A good ROC AUC means the model ranks customers well. It doesn’t mean the win-back offer changes anyone’s mind. To measure that, hold out a random slice of flagged customers (say 20%) who don’t get the offer. After 90 days, compare churn between the contacted and held-out groups. The difference is the effect of the offer on the customers the model picks, and multiplied by customer value it’s the number the business actually cares about. Without the held-out group, any drop in churn could be the season, a competitor’s stumble or luck.

\n\n

Where you are now

\n

You started with a question and a table, and you now have a process that holds up: frame the problem, split before you look, beat a majority-class dummy and a rule, choose metrics by the cost of each mistake, compare models with cross-validation inside leak-free pipelines, check who the model serves worse, and ship something you can monitor and test. The champion turned out to be the simplest serious model, and you know exactly why. That habit of asking “compared to what?” is the most valuable thing this course had to teach, and it applies unchanged to the next model you build, whatever it is.

\n","toc":[{"id":"sec-1","text":"Save the pipeline, not just the model","level":2},{"id":"sec-2","text":"Guard the inputs","level":2},{"id":"sec-3","text":"Batch job or API?","level":2},{"id":"sec-4","text":"Did it work? Run an experiment","level":2},{"id":"sec-5","text":"Where you are now","level":2}],"takeaways":["Save the whole fitted pipeline, not only the model, so preprocessing travels with it; store the feature list, a version and the scikit-learn version alongside.","joblib and pickle files can run code when loaded, so only load model files you created or trust, and load them with the same scikit-learn version.","[object Object]","Many business models, churn included, are best shipped as a scheduled batch job; an online API is for decisions that must happen during a request.","The real test of a model is a controlled experiment: compare outcomes for flagged customers who got the offer with a randomly held-out group who didn’t."],"further":[{"title":"Model persistence","url":"https://scikit-learn.org/stable/model_persistence.html"},{"title":"pickle: security warning","url":"https://docs.python.org/3/library/pickle.html"}],"quiz":[{"q":"

Why save the fitted pipeline rather than only the LogisticRegression step?

","options":[{"text":"The pipeline file is smaller.","why":"It’s slightly larger, if anything; size isn’t the reason."},{"text":"The scaler’s learned means and standard deviations are part of the model; without them, new data would be scaled differently.","why":"Correct. The pipeline guarantees production applies exactly the preprocessing the model was trained with."},{"text":"joblib can’t save a LogisticRegression on its own.","why":"It can save any picklable object; the issue is consistency, not capability."},{"text":"Pipelines load faster than single estimators.","why":"Load time is negligible either way."}],"answer":1},{"q":"

A colleague emails you model.joblib from an unknown source and asks you to evaluate it. What’s the risk?

","options":[{"text":"Loading it can execute arbitrary code on your machine.","why":"Correct. joblib uses pickle, which can run code during loading. Only load files from sources you trust."},{"text":"It might use too much memory to open.","why":"Possible but minor; the serious risk is code execution."},{"text":"It will silently retrain on your data.","why":"Loading doesn’t train anything. The danger is what the file can run while being loaded."}],"answer":0},{"q":"

Marketing contacts every customer the model flags, and churn among them drops from 79% to 60%. Can you credit the model and the offer?

","options":[{"text":"Yes; churn fell after launch, so the system works.","why":"Churn might have fallen anyway (season, a competitor’s problem). Without a comparison group you can’t tell."},{"text":"Yes, as long as test-set ROC AUC was above 0.8.","why":"AUC shows the model ranks well; it says nothing about whether the offer changes behaviour."},{"text":"No, because churn should be measured over 30 days, not 90.","why":"The window is a business definition; the problem is the missing comparison."},{"text":"Not yet; you need a randomly held-out group of flagged customers who got no offer to compare against.","why":"Correct. The difference between the two groups is the effect of the offer."}],"answer":3}],"exercise":{"kind":"python","files":["churn.csv"],"starter":"import io\n\nimport joblib\nimport pandas as pd\nimport sklearn\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.preprocessing import StandardScaler\n\nchurn = pd.read_csv(\"churn.csv\")\nfeatures = [\"tenure_days\", \"orders\", \"total_spent\", \"avg_order_value\",\n \"days_since_last_order\", \"used_coupon\", \"support_tickets\", \"mobile_share\"]\nchampion = make_pipeline(StandardScaler(), LogisticRegression()).fit(churn[features], churn[\"churned\"])\n\ngood = {\"tenure_days\": 400, \"orders\": 2, \"total_spent\": 180.0, \"avg_order_value\": 90.0,\n \"days_since_last_order\": 150, \"used_coupon\": 0, \"support_tickets\": 1, \"mobile_share\": 0.5}\n\n# 1. A bundle with the fitted pipeline, the feature list, a version string and sklearn's version\nbundle = {\"model\": champion}\n\n# 2. Save it with joblib to an in-memory buffer, then load it back into `reloaded`\nbuffer = io.BytesIO()\nreloaded = None\n\n# 3. Validate, then predict. Raise ValueError if any feature is missing,\n# or if orders < 1, or if mobile_share is outside 0..1.\ndef predict_churn(record, bundle):\n row = pd.DataFrame([record])\n return bundle[\"model\"].predict_proba(row)[0, 1]\n","solution":"import io\n\nimport joblib\nimport pandas as pd\nimport sklearn\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.preprocessing import StandardScaler\n\nchurn = pd.read_csv(\"churn.csv\")\nfeatures = [\"tenure_days\", \"orders\", \"total_spent\", \"avg_order_value\",\n \"days_since_last_order\", \"used_coupon\", \"support_tickets\", \"mobile_share\"]\nchampion = make_pipeline(StandardScaler(), LogisticRegression()).fit(churn[features], churn[\"churned\"])\n\ngood = {\"tenure_days\": 400, \"orders\": 2, \"total_spent\": 180.0, \"avg_order_value\": 90.0,\n \"days_since_last_order\": 150, \"used_coupon\": 0, \"support_tickets\": 1, \"mobile_share\": 0.5}\n\nbundle = {\"model\": champion, \"features\": features,\n \"version\": \"churn-2025-09-30.1\", \"sklearn_version\": sklearn.__version__}\n\nbuffer = io.BytesIO()\njoblib.dump(bundle, buffer)\nbuffer.seek(0)\nreloaded = joblib.load(buffer)\n\ndef predict_churn(record, bundle):\n missing = [f for f in bundle[\"features\"] if f not in record]\n if missing:\n raise ValueError(f\"missing fields: {missing}\")\n if record[\"orders\"] < 1:\n raise ValueError(\"orders must be at least 1\")\n if not 0 <= record[\"mobile_share\"] <= 1:\n raise ValueError(\"mobile_share must be between 0 and 1\")\n row = pd.DataFrame([record])[bundle[\"features\"]]\n return float(bundle[\"model\"].predict_proba(row)[0, 1])\n\nprint(round(predict_churn(good, reloaded), 3))\n","checks":[{"id":"bundle","test":"set(bundle) >= {'model', 'features', 'version', 'sklearn_version'} and list(bundle['features']) == features and bundle['sklearn_version'] == sklearn.__version__"},{"id":"reloaded","test":"reloaded is not None and (reloaded['model'].predict_proba(churn[features]) == champion.predict_proba(churn[features])).all()"},{"id":"predicts","test":"isinstance(predict_churn(good, reloaded), float) and abs(predict_churn(good, reloaded) - float(champion.predict_proba(pd.DataFrame([good])[features])[0, 1])) < 1e-12 and abs(predict_churn(dict(reversed(list(good.items()))), reloaded) - predict_churn(good, reloaded)) < 1e-12"},{"id":"missing","test":"exec('try:\\n predict_churn({k: v for k, v in good.items() if k != \"orders\"}, reloaded)\\n _raised = False\\nexcept ValueError:\\n _raised = True', globals()) or _raised"},{"id":"ranges","test":"exec('try:\\n predict_churn({**good, \"mobile_share\": 7}, reloaded)\\n _raised = False\\nexcept ValueError:\\n _raised = True', globals()) or _raised and exec('try:\\n predict_churn({**good, \"orders\": 0}, reloaded)\\n _raised = False\\nexcept ValueError:\\n _raised = True', globals()) or _raised"}],"title":"Package the champion","prompt":"

Get the champion ready to ship.

\n
    \n
  1. bundle: a dict with "model" (the fitted pipeline), "features" (the feature list), "version" (any string) and "sklearn_version" (sklearn.__version__).
    2. \n
  2. Save bundle with joblib.dump into the in-memory buffer, rewind it, and load it back into reloaded.
    3. \n
  3. Finish predict_churn(record, bundle): raise ValueError if any feature is missing, if orders is below 1, or if mobile_share is outside 0–1. Otherwise return the churn probability as a Python float, with columns in the bundle’s feature order (the record’s keys may come in any order).
    4. \n
\n","hints":["joblib.dump(bundle, buffer), then buffer.seek(0) before joblib.load(buffer): loading reads from the current position.","Build missing = [f for f in bundle["features"] if f not in record] and raise if it isn’t empty.","pd.DataFrame([record])[bundle["features"]] fixes the column order; wrap the result of predict_proba(...)[0, 1] in float()."],"explanation":"

The bundle carries everything a production job needs: the pipeline with its fitted scaler, the column order, a version to log with every prediction, and the library version to pin. Reloading gives identical predictions. The guard turns bad input into an explicit error instead of a plausible-looking probability, and selecting bundle["features"] makes the column order independent of how the caller built the record. In a real service the buffer would be a file in a trusted model store.

\n","checkLabels":{"bundle":"bundle holds model, features, version and sklearn_version","reloaded":"reloaded predicts exactly like the original","predicts":"predict_churn returns the right float, whatever the key order","missing":"A missing feature raises ValueError","ranges":"orders below 1 or mobile_share outside 0–1 raise ValueError"}},"words":804,"readMinutes":4}