{"id":"l-pda-2-4","courseId":"python-data-analysis","lang":"en","kind":"lesson","summary":"Filter rows with boolean masks, combine conditions with &amp;, | and ~, use isin and between for common filters, and sort or rank rows with sort_values and nlargest.","html":"<p>The head of growth’s question has a time window: Q4 2025. Before you can explain that quarter, you need to isolate it, and then slice it further: which of those orders used a coupon, which came through the marketplace, which were Black Friday orders. Every one of those is a filter, and pandas filters the same way every time: build a column of <code dir=\"ltr\">True</code>/<code dir=\"ltr\">False</code>, then keep the <code dir=\"ltr\">True</code> rows.</p>\n<h2 id=\"sec-1\" tabindex=\"-1\">Boolean masks</h2>\n<p>A comparison on a column does not return one answer; it returns one answer per row:</p>\n<div class=\"code-block\" dir=\"ltr\" data-lang=\"python\" data-run=\"python\" data-index=\"0\"><div class=\"code-head\"><span class=\"code-title\">Python</span><span class=\"code-actions\"><button type=\"button\" class=\"code-btn code-run\" data-action=\"run\" data-lang=\"python\" data-index=\"0\" aria-label=\"Run this example\"><svg class=\"icon \" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M5 5a2 2 0 0 1 3.008-1.728l11.997 6.998a2 2 0 0 1 .003 3.458l-12 7A2 2 0 0 1 5 19z\"/></svg><span>Run</span></button><button type=\"button\" class=\"code-btn\" data-action=\"copy\" aria-label=\"Copy code\"><svg class=\"icon \" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" aria-hidden=\"true\" focusable=\"false\"><rect width=\"14\" height=\"14\" x=\"8\" y=\"8\" rx=\"2\" ry=\"2\"/><path d=\"M4 16c-1.1 0-2-.9-2-2V4c0-1.1.9-2 2-2h10c1.1 0 2 .9 2 2\"/></svg><span>Copy</span></button></span></div><pre class=\"shiki\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">import</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> pandas </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">as</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> pd</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">orders </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> pd.read_csv(</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"orders.csv\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#953800;--shiki-dark:#FFA657\">parse_dates</span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"order_date\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">])</span></span>\n<span class=\"line\"><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">is_free </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> orders[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"shipping_fee\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">] </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">==</span><span style=\"color:#0550AE;--shiki-dark:#79C0FF\"> 0</span></span>\n<span class=\"line\"><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">print</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">(is_free.head())</span></span>\n<span class=\"line\"><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">print</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">(is_free.sum(), </span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"orders with free shipping\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">)</span></span>\n<span class=\"line\"><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">print</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">(</span><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">round</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">(is_free.mean(), </span><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">3</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">), </span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"share of all orders\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">)</span></span></code></pre><div class=\"run-output\" data-run-output=\"0\" hidden></div></div>\n<p>That boolean Series is called a <strong>mask</strong>. Summing it counts the <code dir=\"ltr\">True</code> values, and its mean is the share of rows that match: 43% of Cartwheel’s orders shipped free. You often learn what you need from the count alone, without filtering anything.</p>\n<p>To keep the matching rows, put the mask inside square brackets, or inside <code dir=\"ltr\">.loc</code> when you also want to choose columns:</p>\n<div class=\"code-block\" dir=\"ltr\" data-lang=\"python\" data-run=\"python\" data-index=\"1\"><div class=\"code-head\"><span class=\"code-title\">Python</span><span class=\"code-actions\"><button type=\"button\" class=\"code-btn code-run\" data-action=\"run\" data-lang=\"python\" data-index=\"1\" aria-label=\"Run this example\"><svg class=\"icon \" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M5 5a2 2 0 0 1 3.008-1.728l11.997 6.998a2 2 0 0 1 .003 3.458l-12 7A2 2 0 0 1 5 19z\"/></svg><span>Run</span></button><button type=\"button\" class=\"code-btn\" data-action=\"copy\" aria-label=\"Copy code\"><svg class=\"icon \" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" aria-hidden=\"true\" focusable=\"false\"><rect width=\"14\" height=\"14\" x=\"8\" y=\"8\" rx=\"2\" ry=\"2\"/><path d=\"M4 16c-1.1 0-2-.9-2-2V4c0-1.1.9-2 2-2h10c1.1 0 2 .9 2 2\"/></svg><span>Copy</span></button></span></div><pre class=\"shiki\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">import</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> pandas </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">as</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> pd</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">orders </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> pd.read_csv(</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"orders.csv\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#953800;--shiki-dark:#FFA657\">parse_dates</span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"order_date\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">])</span></span>\n<span class=\"line\"><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">cancelled </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> orders[orders[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"status\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">] </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">==</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\"> \"cancelled\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">]</span></span>\n<span class=\"line\"><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">print</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">(</span><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">len</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">(cancelled))</span></span>\n<span class=\"line\"><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">print</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">(orders.loc[orders[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"shipping_fee\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">] </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">==</span><span style=\"color:#0550AE;--shiki-dark:#79C0FF\"> 0</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, [</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"order_id\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"channel\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"coupon_code\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">]].head())</span></span></code></pre><div class=\"run-output\" data-run-output=\"1\" hidden></div></div>\n<h2 id=\"sec-2\" tabindex=\"-1\">Combining conditions</h2>\n<p>Real questions have several conditions. pandas uses <code dir=\"ltr\">&amp;</code> for and, <code dir=\"ltr\">|</code> for or, and <code dir=\"ltr\">~</code> for not, applied row by row:</p>\n<div class=\"code-block\" dir=\"ltr\" data-lang=\"python\" data-run=\"python\" data-index=\"2\"><div class=\"code-head\"><span class=\"code-title\">Python</span><span class=\"code-actions\"><button type=\"button\" class=\"code-btn code-run\" data-action=\"run\" data-lang=\"python\" data-index=\"2\" aria-label=\"Run this example\"><svg class=\"icon \" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M5 5a2 2 0 0 1 3.008-1.728l11.997 6.998a2 2 0 0 1 .003 3.458l-12 7A2 2 0 0 1 5 19z\"/></svg><span>Run</span></button><button type=\"button\" class=\"code-btn\" data-action=\"copy\" aria-label=\"Copy code\"><svg class=\"icon \" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" aria-hidden=\"true\" focusable=\"false\"><rect width=\"14\" height=\"14\" x=\"8\" y=\"8\" rx=\"2\" ry=\"2\"/><path d=\"M4 16c-1.1 0-2-.9-2-2V4c0-1.1.9-2 2-2h10c1.1 0 2 .9 2 2\"/></svg><span>Copy</span></button></span></div><pre class=\"shiki\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">import</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> pandas </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">as</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> pd</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">orders </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> pd.read_csv(</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"orders.csv\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#953800;--shiki-dark:#FFA657\">parse_dates</span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"order_date\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">])</span></span>\n<span class=\"line\"><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">in_q4 </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> (orders[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"order_date\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">] </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">>=</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\"> \"2025-10-01\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">) </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">&#x26;</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> (orders[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"order_date\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">] </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">&#x3C;</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\"> \"2026-01-01\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">)</span></span>\n<span class=\"line\"><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">not_cancelled </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> orders[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"status\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">] </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">!=</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\"> \"cancelled\"</span></span>\n<span class=\"line\"><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">q4 </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> orders[in_q4 </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">&#x26;</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> not_cancelled]</span></span>\n<span class=\"line\"><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">print</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">(</span><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">len</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">(orders[in_q4]), </span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"orders in Q4 2025,\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">len</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">(q4), </span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"not cancelled\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">)</span></span>\n<span class=\"line\"><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">print</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">(q4[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"channel\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">].value_counts())</span></span></code></pre><div class=\"run-output\" data-run-output=\"2\" hidden></div></div>\n<p>Giving each mask a name, as here, keeps long filters readable and lets you print and count each condition separately when a result looks wrong. A datetime column can be compared with a date written as a string, <code dir=\"ltr\">&quot;2025-10-01&quot;</code>, and pandas converts it for you. The upper bound is “before 1 January”, not “on or before 31 December”, because orders on 31 December have a time later than midnight.</p>\n<aside class=\"callout callout-mistake\" role=\"note\"><p class=\"callout-title\"><svg class=\"icon \" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" aria-hidden=\"true\" focusable=\"false\"><path d=\"m21.73 18-8-14a2 2 0 0 0-3.48 0l-8 14A2 2 0 0 0 4 21h16a2 2 0 0 0 1.73-3\"/><path d=\"M12 9v4\"/><path d=\"M12 17h.01\"/></svg><span><span class=\"callout-kind\">Common mistake</span><span class=\"callout-sep\" aria-hidden=\"true\"> · </span>and, or, missing parentheses</span></p>\n<p><code dir=\"ltr\">orders[a == 1 and b == 2]</code> raises <code dir=\"ltr\">ValueError: The truth value of a Series is ambiguous</code>. Python’s <code dir=\"ltr\">and</code> wants one True or False, and a Series has many. Use <code dir=\"ltr\">&amp;</code>. And wrap each condition in parentheses: <code dir=\"ltr\">&amp;</code> binds more tightly than <code dir=\"ltr\">==</code>, so <code dir=\"ltr\">orders[&quot;channel&quot;] == &quot;web&quot; &amp; orders[&quot;shipping_fee&quot;] == 0</code> is evaluated in the wrong order and fails.</p>\n</aside>\n<h2 id=\"sec-3\" tabindex=\"-1\">Two shortcuts: isin and between</h2>\n<p>Matching any of several values with <code dir=\"ltr\">|</code> chains gets long. <code dir=\"ltr\">isin</code> takes a list:</p>\n<div class=\"code-block\" dir=\"ltr\" data-lang=\"python\" data-run=\"python\" data-index=\"3\"><div class=\"code-head\"><span class=\"code-title\">Python</span><span class=\"code-actions\"><button type=\"button\" class=\"code-btn code-run\" data-action=\"run\" data-lang=\"python\" data-index=\"3\" aria-label=\"Run this example\"><svg class=\"icon \" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M5 5a2 2 0 0 1 3.008-1.728l11.997 6.998a2 2 0 0 1 .003 3.458l-12 7A2 2 0 0 1 5 19z\"/></svg><span>Run</span></button><button type=\"button\" class=\"code-btn\" data-action=\"copy\" aria-label=\"Copy code\"><svg class=\"icon \" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" aria-hidden=\"true\" focusable=\"false\"><rect width=\"14\" height=\"14\" x=\"8\" y=\"8\" rx=\"2\" ry=\"2\"/><path d=\"M4 16c-1.1 0-2-.9-2-2V4c0-1.1.9-2 2-2h10c1.1 0 2 .9 2 2\"/></svg><span>Copy</span></button></span></div><pre class=\"shiki\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">import</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> pandas </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">as</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> pd</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">orders </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> pd.read_csv(</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"orders.csv\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#953800;--shiki-dark:#FFA657\">parse_dates</span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"order_date\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">])</span></span>\n<span class=\"line\"><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">problem </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> orders[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"status\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">].isin([</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"cancelled\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"returned\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">])</span></span>\n<span class=\"line\"><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">print</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">(problem.sum(), </span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"cancelled or returned\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">)</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">black_friday_week </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> orders[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"order_date\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">].between(</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"2025-11-24\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"2025-11-30 23:59:59\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">)</span></span>\n<span class=\"line\"><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">print</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">(black_friday_week.sum(), </span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"orders in Black Friday week 2025\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">)</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">bf </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> orders[orders[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"coupon_code\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">] </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">==</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\"> \"BLACKFRIDAY25\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">]</span></span>\n<span class=\"line\"><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">print</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">(bf[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"order_date\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">].dt.month.value_counts().head(</span><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">3</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">))</span></span></code></pre><div class=\"run-output\" data-run-output=\"3\" hidden></div></div>\n<p><code dir=\"ltr\">between</code> includes both ends, which is why the upper bound carries a time. The last lines contain a real finding: of 124 orders with the <code dir=\"ltr\">BLACKFRIDAY25</code> coupon, 91 are in November, and the rest are scattered across the year. Was the code left active, shared on a coupon site, or is the data wrong? Filters do not answer that, but they surface the question, and good analysts take it to the people who run the promotion before building on it. (<code dir=\"ltr\">.dt.month</code> extracts the month number; Lesson 4.4 covers the <code dir=\"ltr\">dt</code> accessor properly.)</p>\n<h3 id=\"sec-4\" tabindex=\"-1\">A filtered table is a new table</h3>\n<p>A filter returns a new DataFrame. <code dir=\"ltr\">q4</code> above does not remember that it came from <code dir=\"ltr\">orders</code>, and you should treat it as its own table: if you later want to add a column to it, you add it to <code dir=\"ltr\">q4</code>, and <code dir=\"ltr\">orders</code> stays untouched. In pandas 2 code you will often see <code dir=\"ltr\">.copy()</code> after a filter: it makes that explicit and silences the <code dir=\"ltr\">SettingWithCopyWarning</code> pandas 2 prints when you add a column to a filtered table. With Copy-on-Write, which is the default from pandas 3.0 and which you can switch on in pandas 2 with <code dir=\"ltr\">pd.options.mode.copy_on_write = True</code>, every filtered result already behaves as an independent copy. Lesson 2.5 shows why this matters when you start changing values.</p>\n<h2 id=\"sec-5\" tabindex=\"-1\">Sorting and top-n</h2>\n<p><code dir=\"ltr\">sort_values</code> orders rows by one or more columns:</p>\n<div class=\"code-block\" dir=\"ltr\" data-lang=\"python\" data-run=\"python\" data-index=\"4\"><div class=\"code-head\"><span class=\"code-title\">Python</span><span class=\"code-actions\"><button type=\"button\" class=\"code-btn code-run\" data-action=\"run\" data-lang=\"python\" data-index=\"4\" aria-label=\"Run this example\"><svg class=\"icon \" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M5 5a2 2 0 0 1 3.008-1.728l11.997 6.998a2 2 0 0 1 .003 3.458l-12 7A2 2 0 0 1 5 19z\"/></svg><span>Run</span></button><button type=\"button\" class=\"code-btn\" data-action=\"copy\" aria-label=\"Copy code\"><svg class=\"icon \" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" aria-hidden=\"true\" focusable=\"false\"><rect width=\"14\" height=\"14\" x=\"8\" y=\"8\" rx=\"2\" ry=\"2\"/><path d=\"M4 16c-1.1 0-2-.9-2-2V4c0-1.1.9-2 2-2h10c1.1 0 2 .9 2 2\"/></svg><span>Copy</span></button></span></div><pre class=\"shiki\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">import</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> pandas </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">as</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> pd</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">orders </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> pd.read_csv(</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"orders.csv\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#953800;--shiki-dark:#FFA657\">parse_dates</span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"order_date\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">])</span></span>\n<span class=\"line\"><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">latest </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> orders.sort_values(</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"order_date\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#953800;--shiki-dark:#FFA657\">ascending</span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">False</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">)</span></span>\n<span class=\"line\"><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">print</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">(latest[[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"order_id\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"order_date\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"status\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">]].head(</span><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">3</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">))</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">by_fee </span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\"> orders.sort_values([</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"shipping_fee\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"order_date\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">], </span><span style=\"color:#953800;--shiki-dark:#FFA657\">ascending</span><span style=\"color:#CF222E;--shiki-dark:#FF7B72\">=</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">[</span><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">False</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">False</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">])</span></span>\n<span class=\"line\"><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">print</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">(by_fee[[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"order_id\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"shipping_fee\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"order_date\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">]].head(</span><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">3</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">))</span></span>\n<span class=\"line\"><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">print</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">(orders.nlargest(</span><span style=\"color:#0550AE;--shiki-dark:#79C0FF\">3</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"shipping_fee\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">)[[</span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"order_id\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">, </span><span style=\"color:#0A3069;--shiki-dark:#A5D6FF\">\"shipping_fee\"</span><span style=\"color:#1F2328;--shiki-dark:#E6EDF3\">]])</span></span></code></pre><div class=\"run-output\" data-run-output=\"4\" hidden></div></div>\n<p>With a list of columns, the second column breaks ties in the first, and <code dir=\"ltr\">ascending</code> takes a matching list. <code dir=\"ltr\">nlargest(n, column)</code> and <code dir=\"ltr\">nsmallest</code> are shortcuts for “sort and take the top n”; they are faster on big tables and say what you mean.</p>\n<p>The three latest orders are still <code dir=\"ltr\">processing</code> or <code dir=\"ltr\">shipped</code>: they were placed in the last days of December and had not been delivered when the data was exported. That is another decision for your revenue definition, which you will make in Section 4.</p>\n<aside class=\"callout callout-tip\" role=\"note\"><p class=\"callout-title\"><svg class=\"icon \" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M15 14c.2-1 .7-1.7 1.5-2.5 1-.9 1.5-2.2 1.5-3.5A6 6 0 0 0 6 8c0 1 .2 2.2 1.5 3.5.7.7 1.3 1.5 1.5 2.5\"/><path d=\"M9 18h6\"/><path d=\"M10 22h4\"/></svg><span><span class=\"callout-kind\">Tip</span><span class=\"callout-sep\" aria-hidden=\"true\"> · </span>Count before and after</span></p>\n<p>Print <code dir=\"ltr\">len(df)</code> before and after every filter. If a filter that should remove a handful of rows removes half the table, or none at all, you find out immediately instead of three steps later.</p>\n</aside>\n<p>In the exercise you isolate Q4 2025 and measure its coupon use. Next you add new columns, such as each line’s revenue, and learn the one correct way to change values in a DataFrame.</p>\n","toc":[{"id":"sec-1","text":"Boolean masks","level":2},{"id":"sec-2","text":"Combining conditions","level":2},{"id":"sec-3","text":"Two shortcuts: isin and between","level":2},{"id":"sec-4","text":"A filtered table is a new table","level":3},{"id":"sec-5","text":"Sorting and top-n","level":2}],"takeaways":["A comparison on a column returns a boolean mask; <code dir=\"ltr\">df[mask]</code> or <code dir=\"ltr\">df.loc[mask, cols]</code> keeps the rows where it is <code dir=\"ltr\">True</code>.","Combine masks with <code dir=\"ltr\">&amp;</code> (and), <code dir=\"ltr\">|</code> (or) and <code dir=\"ltr\">~</code> (not), and wrap every condition in parentheses.","<code dir=\"ltr\">mask.sum()</code> counts matching rows and <code dir=\"ltr\">mask.mean()</code> gives their share, without filtering anything.","<code dir=\"ltr\">isin([...])</code> replaces long chains of <code dir=\"ltr\">==</code> with <code dir=\"ltr\">|</code>, and <code dir=\"ltr\">between(a, b)</code> includes both ends.","<code dir=\"ltr\">sort_values</code> takes lists for multi-column sorts; <code dir=\"ltr\">nlargest(n, col)</code> is the shortcut for a top-n list."],"further":[{"title":"Boolean indexing","url":"https://pandas.pydata.org/docs/user_guide/indexing.html#boolean-indexing"},{"title":"pandas.DataFrame.sort_values","url":"https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.sort_values.html"},{"title":"pandas.Series.isin","url":"https://pandas.pydata.org/docs/reference/api/pandas.Series.isin.html"}],"quiz":[{"q":"<p>Which line keeps web orders that used a coupon?</p>","options":[{"text":"<code dir=\"ltr\">orders[(orders[&quot;channel&quot;] == &quot;web&quot;) &amp; (orders[&quot;coupon_code&quot;].notna())]</code>","why":"Correct. Each condition is a boolean Series in its own parentheses, combined element by element with <code dir=\"ltr\">&amp;</code>."},{"text":"<code dir=\"ltr\">orders[orders[&quot;channel&quot;] == &quot;web&quot; and orders[&quot;coupon_code&quot;].notna()]</code>","why":"<code dir=\"ltr\">and</code> asks for a single True/False of a whole Series, so pandas raises <code dir=\"ltr\">ValueError: The truth value of a Series is ambiguous</code>."},{"text":"<code dir=\"ltr\">orders[orders[&quot;channel&quot;] == &quot;web&quot; &amp; orders[&quot;coupon_code&quot;].notna()]</code>","why":"Without parentheses, <code dir=\"ltr\">&amp;</code> binds before <code dir=\"ltr\">==</code>, so Python evaluates <code dir=\"ltr\">&quot;web&quot; &amp; ...</code> first and the expression fails or means something else."},{"text":"<code dir=\"ltr\">orders[&quot;channel&quot; == &quot;web&quot;][&quot;coupon_code&quot;]</code>","why":"<code dir=\"ltr\">&quot;channel&quot; == &quot;web&quot;</code> compares two strings and is always <code dir=\"ltr\">False</code>; it never looks at the column."}],"answer":0},{"q":"<p><code dir=\"ltr\">mask = orders[&quot;shipping_fee&quot;] == 0</code>. What does <code dir=\"ltr\">mask.mean()</code> return?</p>","options":[{"text":"The average shipping fee","why":"That would be <code dir=\"ltr\">orders[&quot;shipping_fee&quot;].mean()</code>. The mask holds booleans, not fees."},{"text":"The number of free-shipping orders","why":"That is <code dir=\"ltr\">mask.sum()</code>. The mean divides that count by the number of rows."},{"text":"The share of orders with free shipping, between 0 and 1","why":"Correct. <code dir=\"ltr\">True</code> counts as 1 and <code dir=\"ltr\">False</code> as 0, so the mean of a mask is the fraction of rows where it is true."}],"answer":2},{"q":"<p>You want the 10 orders with the highest shipping fee, newest first among equal fees. Which call is right?</p>","options":[{"text":"<code dir=\"ltr\">orders.sort_values(&quot;shipping_fee&quot;).head(10)</code>","why":"The default sort is ascending, so this gives the ten cheapest, and ties are not ordered by date."},{"text":"<code dir=\"ltr\">orders.sort_values([&quot;shipping_fee&quot;, &quot;order_date&quot;], ascending=[False, True]).head(10)</code>","why":"The date order is ascending here, which puts the oldest orders first among equal fees."},{"text":"<code dir=\"ltr\">orders.nlargest(10, &quot;order_date&quot;)</code>","why":"This ranks by date, not by fee."},{"text":"<code dir=\"ltr\">orders.sort_values([&quot;shipping_fee&quot;, &quot;order_date&quot;], ascending=[False, False]).head(10)</code>","why":"Correct. Both columns sort descending, fee first, then date to break ties."}],"answer":3}],"exercise":{"kind":"python","files":["orders.csv"],"starter":"import pandas as pd\n\norders = pd.read_csv(\"orders.csv\", parse_dates=[\"order_date\"])\n\n# 1. Orders placed in Q4 2025 (October to December) that were NOT cancelled\nq4 = None\n\n# 2. Share of those Q4 orders that used any coupon, rounded to 3 decimals\ncoupon_share = None\n\n# 3. In the whole table: number of marketplace orders that used a coupon\nmarketplace_coupons = None\n\n# 4. order_id of the 3 most recent orders in q4, newest first, as a list\nlatest_ids = None\n","solution":"import pandas as pd\n\norders = pd.read_csv(\"orders.csv\", parse_dates=[\"order_date\"])\n\nin_q4 = (orders[\"order_date\"] >= \"2025-10-01\") & (orders[\"order_date\"] < \"2026-01-01\")\nq4 = orders[in_q4 & (orders[\"status\"] != \"cancelled\")]\nprint(len(q4))\n\ncoupon_share = round(q4[\"coupon_code\"].notna().mean(), 3)\n\nmarketplace_coupons = ((orders[\"channel\"] == \"marketplace\") & orders[\"coupon_code\"].notna()).sum()\n\nlatest_ids = q4.nlargest(3, \"order_date\")[\"order_id\"].tolist()\n","checks":[{"id":"q4-rows","test":"isinstance(q4, pd.DataFrame) and len(q4) == 642"},{"id":"q4-clean","test":"(q4['status'] != 'cancelled').all() and q4['order_date'].min() >= pd.Timestamp('2025-10-01')"},{"id":"coupon-share","test":"round(float(coupon_share), 3) == 0.231"},{"id":"marketplace","test":"int(marketplace_coupons) == 44"},{"id":"latest","test":"list(latest_ids) == [2017, 2735, 1550]"}],"title":"Isolate Q4 2025","prompt":"<p>Using <code dir=\"ltr\">orders</code> (with <code dir=\"ltr\">order_date</code> already parsed): set <code dir=\"ltr\">q4</code> to the orders placed from 1 October to 31 December 2025 that were not cancelled. Set <code dir=\"ltr\">coupon_share</code> to the share of <code dir=\"ltr\">q4</code> orders that used a coupon, rounded to 3 decimals. Set <code dir=\"ltr\">marketplace_coupons</code> to the number of orders in the whole table that came through the <code dir=\"ltr\">&quot;marketplace&quot;</code> channel and used a coupon. Set <code dir=\"ltr\">latest_ids</code> to a list of the <code dir=\"ltr\">order_id</code>s of the 3 most recent orders in <code dir=\"ltr\">q4</code>, newest first.</p>\n","hints":["Compare <code dir=\"ltr\">order_date</code> with <code dir=\"ltr\">&quot;2025-10-01&quot;</code> and <code dir=\"ltr\">&quot;2026-01-01&quot;</code>, and combine masks with <code dir=\"ltr\">&amp;</code>, each in parentheses.","A coupon was used when <code dir=\"ltr\">coupon_code</code> is not missing: <code dir=\"ltr\">.notna()</code>. The mean of a boolean Series is a share; <code dir=\"ltr\">.sum()</code> is a count.","<code dir=\"ltr\">q4.nlargest(3, &quot;order_date&quot;)[&quot;order_id&quot;].tolist()</code> gives the newest three ids as a list."],"explanation":"<p>Q4 2025 has 677 orders, of which 642 were not cancelled. Using “before 1 January” as the upper bound keeps every order on 31 December, whatever its time. 23.1% of those orders used a coupon, above the 18.8% average across both years, an early hint that promotions matter to the Q4 story. Counting with <code dir=\"ltr\">(mask_a &amp; mask_b).sum()</code> answers the marketplace question without building a filtered table. For the latest ids, <code dir=\"ltr\">nlargest</code> on the date column (or <code dir=\"ltr\">sort_values(&quot;order_date&quot;, ascending=False).head(3)</code>) both work.</p>\n","checkLabels":{"q4-rows":"<code dir=\"ltr\">q4</code> has 642 orders","q4-clean":"<code dir=\"ltr\">q4</code> has no cancelled orders and nothing before October 2025","coupon-share":"<code dir=\"ltr\">coupon_share</code> is 0.231","marketplace":"44 marketplace orders used a coupon","latest":"<code dir=\"ltr\">latest_ids</code> is <code dir=\"ltr\">[2017, 2735, 1550]</code>"}},"words":718,"readMinutes":3}