Large language models have changed what software can do with text, and every data team now gets asked “can’t we just use an LLM for this?”. Sometimes the answer is yes. Often it’s “for part of it”. To answer well, you need an accurate picture of what LLMs and embeddings are, and that picture is built from ideas you’ve used all course.
\nA large language model is a neural network, almost always a transformer, trained on an enormous amount of text with one objective: given the text so far, predict the next token (a word or a piece of a word). That’s pretraining. Done at sufficient scale, next-token prediction forces the network to absorb grammar, facts, styles and a surprising amount of reasoning-like behaviour, because all of it helps predict what comes next.
\nA pretrained model then gets further training to follow instructions and to give helpful, safe answers, usually with human-written examples and human or AI preference feedback. When you send it a prompt, it computes a probability for every possible next token, picks one (with some randomness, controlled by a temperature setting), appends it, and repeats.
\nThree consequences matter for a data scientist:
\npredict_proba was in section 3.The more quietly useful idea is the embedding. An embedding model reads a piece of text and returns a fixed-length vector, often several hundred to a few thousand numbers, arranged so that texts with similar meaning get nearby vectors. Cartwheel’s support tickets become rows of numbers, and everything you know about rows of numbers applies.
\nCloseness is measured with cosine similarity: the cosine of the angle between two vectors, 1 for the same direction, near 0 for unrelated. Compare that with the word-count features scikit-learn can build on its own:
\nimport numpy as np\nfrom sklearn.feature_extraction.text import TfidfVectorizer\nfrom sklearn.metrics.pairwise import cosine_similarity\n\ntickets = [\"My parcel never arrived\", \"The package did not show up\",\n \"I was charged twice for one order\", \"Refund the duplicate payment please\",\n \"How do I change my delivery address\"]\n\ntfidf = TfidfVectorizer().fit_transform(tickets)\nprint(\"TF-IDF similarity of ticket 0 to the others:\", cosine_similarity(tfidf)[0].round(2))\n\n# Illustrative 4-number \"embeddings\" (real ones come from an embedding model and are far longer)\nemb = np.array([[0.90, 0.10, 0.00, 0.20], [0.85, 0.15, 0.05, 0.10],\n [0.05, 0.90, 0.10, 0.00], [0.10, 0.85, 0.20, 0.05],\n [0.30, 0.00, 0.10, 0.90]])\nprint(\"embedding similarity of ticket 0 to the others:\", cosine_similarity(emb)[0].round(2))TF-IDF counts shared words. “Parcel never arrived” and “package did not show up” share none, so it scores them 0, yet it links ticket 0 to the address question because both contain “my”. Embeddings score the two delivery complaints as near-identical. The vectors here are hand-made to show the idea; a real embedding model produces this behaviour from its training, for texts it has never seen.
\n\n| Task | \nGood fit | \nWhy | \n
|---|---|---|
| Predict churn from customer facts | \nscikit-learn on the table | \nLabelled tabular data, needs calibrated, cheap, testable scores | \n
| Route support tickets to teams | \nEmbeddings + logistic regression, or an LLM | \nText input; embeddings make it a classic classifier | \n
| Find tickets similar to a new one | \nEmbeddings + nearest neighbours | \nSimilarity search is what embeddings are for | \n
| Summarise a week of complaints for a manager | \nAn LLM | \nFree-form text output; a person reviews it | \n
| Add “complaint topic” as a churn feature | \nEmbeddings → k-means, or an LLM label | \nTurns text into a column the churn model can test | \n
The last row is where the two worlds meet. Text becomes a feature; cross-validation decides whether it helps. Nothing about the discipline changes because the feature came from a large model.
\n\nWhichever model you ship, it will make decisions about people. Next lesson asks whether those decisions are equally good for everyone, and what to watch after launch.
\n","toc":[{"id":"sec-1","text":"What an LLM is","level":2},{"id":"sec-2","text":"Embeddings: text as coordinates","level":2},{"id":"sec-3","text":"Where each tool fits at Cartwheel","level":2}],"takeaways":["A large language model is a transformer network pretrained to predict the next token on huge amounts of text, then tuned to follow instructions.","An LLM’s output is sampled text, not a calibrated probability, so it needs the same evaluation discipline as any model: a labelled test set and a baseline.","An embedding model maps a text to a fixed-length vector so that texts with similar meaning land close together; cosine similarity measures how close.","Embeddings turn unstructured text into features that classic tools (k-NN, logistic regression, k-means) can use.","For tabular predictions such as churn, scikit-learn models stay cheaper, faster and easier to validate; LLMs and embeddings earn their place on text."],"further":[{"title":"Feature extraction from text","url":"https://scikit-learn.org/stable/modules/feature_extraction.html#text-feature-extraction"},{"title":"cosine_similarity","url":"https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.cosine_similarity.html"}],"quiz":[{"q":"What is a large language model fundamentally trained to do during pretraining?
","options":[{"text":"Look up answers in a database of verified facts.","why":"LLMs don’t consult a fact database by default; they generate text from patterns learned in their weights."},{"text":"Classify documents into a fixed set of topics.","why":"Classification can be done with an LLM, but it isn’t the pretraining objective."},{"text":"Translate between languages only.","why":"Translation is one emergent ability, not the training objective."},{"text":"Predict the next token of text, over and over, on a huge corpus.","why":"Correct. Instruction following and chat behaviour come from further tuning on top of that."}],"answer":3},{"q":"TF-IDF gives ‘My parcel never arrived’ and ‘The package did not show up’ a cosine similarity of 0. An embedding model gives them about 0.9. Why the difference?
","options":[{"text":"TF-IDF only matches shared words, and these two share none; embeddings place texts by meaning.","why":"Correct. Synonyms and paraphrases are exactly where embeddings beat word counts."},{"text":"TF-IDF is broken for short texts.","why":"It works as designed; it just has no notion that ‘parcel’ and ‘package’ mean the same thing."},{"text":"The embedding model memorised these two sentences.","why":"Embedding models generalise from training on huge text collections; they don’t need to have seen these exact sentences."}],"answer":0},{"q":"Cartwheel wants to predict churn from the eight numeric columns. A colleague suggests sending each customer’s row to an LLM and asking ‘will they churn?’. What’s the strongest objection?
","options":[{"text":"LLMs can’t read numbers.","why":"They can read numbers in text; the problems are calibration, cost and validation, not reading."},{"text":"It would be less accurate than any baseline by definition.","why":"Nothing guarantees that; you’d have to measure. The point is you’d pay more for something no better validated."},{"text":"The task already has labelled tabular data, where a scikit-learn model is cheaper, faster, testable and gives real probabilities.","why":"Correct. Use the tool that fits the data; an LLM adds cost and uncertainty with no clear gain here."},{"text":"LLMs are only allowed for chat applications.","why":"There’s no such rule; LLMs are used for many tasks. The question is fit, not permission."}],"answer":2}],"exercise":{"kind":"choice","multiple":true,"options":["review-search","refund-forecast","ticket-topic-feature","llm-churn-score","review-summary","coupon-ab"],"correct":["review-search","ticket-topic-feature","review-summary"],"title":"Which tasks call for embeddings or an LLM?","prompt":"Cartwheel’s data team has six requests. Select every one where embeddings or an LLM is a better first tool than a classic scikit-learn model on the existing tables.
\n","hints":["Ask what the input is: a table of facts, or free text?","Ask what the output is: a calibrated number, or text or similarity?","Three requests involve free text; the other three are tabular or are experiments."],"explanation":"Searching reviews by meaning, turning ticket text into a topic feature, and summarising reviews are all text problems: embeddings handle the first two cheaply, and an LLM is the natural tool for a written summary a person will read. Forecasting refunds and scoring churn are tabular predictions where scikit-learn gives cheap, testable, calibrated outputs. The coupon question is an experiment to analyse, not a prediction task at all.
\n","checkLabels":{},"optionText":{"review-search":{"text":"Let staff type “arrived broken” and find product reviews that describe damage in any wording.","why":"Good fit: embeddings + nearest-neighbour search match meaning, not exact words."},"refund-forecast":{"text":"Forecast next month’s refund amount from order and return tables.","why":"Tabular regression with numeric history; a scikit-learn model (with a time-based split) fits."},"ticket-topic-feature":{"text":"Turn free-text support tickets into a ‘complaint topic’ column to test as a churn feature.","why":"Good fit: embed the tickets, cluster or classify them, then let cross-validation judge the new column."},"llm-churn-score":{"text":"Ask a chat model to rate each customer’s churn risk from their eight numeric columns.","why":"The labelled table already supports a cheaper, calibrated, testable model; the LLM adds cost and uncertainty."},"review-summary":{"text":"Write a one-paragraph weekly summary of new product reviews for the merchandising team.","why":"Good fit: generating readable text from text is what LLMs do; a person reviews the result."},"coupon-ab":{"text":"Decide whether the SPRING15 coupon increased order value, using a randomised test.","why":"That’s experiment analysis (compare groups, check significance), not a modelling problem for an LLM."}}},"words":748,"readMinutes":4}