Checkpoint: What Machine Learning Is

Five questions on this section. Answer all of them, then see your score and why each answer is right or wrong.

A churn model scores 0.91 accuracy on the 298 customers it was trained on and 0.70 on the 100 held-out customers. The 80-day rule scores 0.72 on the same 100. What should you tell the team?

The model is excellent; 0.91 is the number to report.
The model is slightly better than the rule because its average of the two scores is higher.
The model needs more features before it can be evaluated.
The model overfits and doesn’t beat the one-line rule on unseen customers, so it isn’t ready to replace it.

Your data has 10,000 rows and the positive class is 3%. You call train_test_split(X, y, test_size=0.2, random_state=0). What risk does leaving out stratify=y create?

The split becomes non-reproducible.
The model can’t be trained on imbalanced data without it.
The test set’s share of positives can drift from 3%, making its metrics less representative.
The training set will contain duplicates of the test rows.

What does this code print?

Python

from sklearn.dummy import DummyClassifier
import numpy as np
X = np.zeros((10, 1))
y = np.array([1, 1, 1, 0, 0, 0, 0, 0, 0, 0])
print(DummyClassifier(strategy="most_frequent").fit(X, y).score(X, y))

0.7
0.3
0.5
An error, because X has only zeros.

You fit model = KNeighborsClassifier().fit(X_train, y_train) and later call model.fit(X_new, y_new) on a second batch of data. What does the model now know?

Both batches, because fit adds new data to what it already learned.
Nothing, because a model can only be fitted once.
The first batch only; the second call is ignored.
Only the second batch, because calling fit again discards what was learned before.

Which question is framed well enough to start a supervised learning project?

Use AI to reduce churn.
On the first of each month, predict for each active customer whether they will place no order in the next 90 days, so marketing can choose who gets an offer.
Find out everything that causes churn.
Build the most accurate model possible on churn.csv.

5 questions unanswered