{"id":"l-dk-5-3","courseId":"docker-kubernetes","lang":"en","kind":"lesson","summary":"Tune notes-api’s rolling update so releases never reduce capacity, watch and roll back a bad rollout with kubectl, and package the manifests as a Helm chart with per-environment values.","html":"

The release that taught me to respect rollouts went out on a Friday at 4 p.m. (I know). The new version had a typo in an environment variable name, so it couldn’t reach the database. The Deployment had no readiness probe and the default rollout settings, so Kubernetes happily replaced every working Pod with one that returned 500s. It took six minutes to notice and forty seconds to roll back. This lesson is about making the first part impossible and the second part routine.

\n

How a rolling update works

\n

When you change anything in a Deployment’s Pod template, typically the image, the Deployment creates a new ReplicaSet for the new template and starts shifting replicas from old to new. Two settings pace it:

\n
k8s/deployment.yaml
spec:\n  revisionHistoryLimit: 10\n  minReadySeconds: 10\n  progressDeadlineSeconds: 300\n  strategy:\n    type: RollingUpdate\n    rollingUpdate:\n      maxSurge: 1\n      maxUnavailable: 0
\n\n

The defaults are 25% surge and 25% unavailable. I set maxUnavailable: 0 on every user-facing service: an old Pod is removed only after a new Pod is ready. Combined with the readiness probe from lesson 5.1, a broken release can’t take traffic away from a working one. It stalls with one unready new Pod while the old ones serve.

\n
\n Four steps of a rolling update for 4 replicas. Each step adds one new-version Pod, waits for it to be ready, then removes one old-version Pod, so at least 4 ready Pods always exist.\n start\n \n \n \n \n surge\n \n \n \n \n \n new Pod ready\n swap\n \n \n \n \n one old Pod removed\n done\n \n \n \n \n old ReplicaSet at 0\n grey: 1.4.0 · green: 1.5.0 · repeat until all replaced\n
Rolling update with maxSurge 1 and maxUnavailable 0
\n

The other strategy, Recreate, deletes all old Pods before starting new ones. Use it only when two versions truly can’t run side by side, and accept the downtime.

\n

Deploy, watch, roll back

\n

Change the image in k8s/deployment.yaml and apply, then watch:

\n
Terminal
kubectl apply -f k8s/deployment.yaml\nkubectl rollout status deployment/notes-api --timeout=5m
\n

rollout status waits until the rollout completes or fails, and exits non-zero on failure, so CI can use it as the “did the deploy work?” step. If it fails, or your dashboards turn red after it succeeds:

\n
Terminal
kubectl rollout history deployment/notes-api\nkubectl rollout undo deployment/notes-api\nkubectl rollout undo deployment/notes-api --to-revision=7
\n

Every old ReplicaSet (up to revisionHistoryLimit) is a ready-made rollback target. undo scales the previous one back up using the same rolling update rules, so rolling back is as safe as rolling forward and takes seconds. To make history readable, record why each revision exists: kubectl annotate deployment/notes-api kubernetes.io/change-cause="1.5.0: bulk export".

\n

After an undo, the cluster runs 1.4.0 and Git still says 1.5.0. Revert the commit straight away, or the next routine apply redeploys the bug.

\n\n

Packaging with Helm

\n

notes-api’s k8s/ folder is now six files, and staging needs different replicas, resources and image tags from production. Helm packages manifests into a chart: templates with placeholders, plus a values.yaml of defaults that each environment overrides.

\n
Text
charts/notes-api/\n  Chart.yaml\n  values.yaml\n  templates/\n    deployment.yaml\n    service.yaml\n    hpa.yaml\n    configmap.yaml
\n
charts/notes-api/templates/deployment.yaml
      containers:\n        - name: api\n          image: \"{{ .Values.image.repository }}:{{ .Values.image.tag }}\"\n          resources:\n            {{- toYaml .Values.resources | nindent 12 }}
\n
charts/notes-api/values-prod.yaml
image:\n  repository: ghcr.io/skylane/notes-api\n  tag: \"1.5.1\"\nresources:\n  requests: { cpu: 250m, memory: 256Mi }\n  limits: { memory: 512Mi }
\n

The commands mirror what you did with kubectl:

\n
Terminal
helm lint charts/notes-api\nhelm template notes-api charts/notes-api -f charts/notes-api/values-prod.yaml\nhelm upgrade --install notes-api charts/notes-api -n notes -f charts/notes-api/values-prod.yaml\nhelm history notes-api -n notes\nhelm rollback notes-api 4 -n notes
\n

helm template renders the YAML locally so you can review it. upgrade --install installs the first time and upgrades after that, recording a numbered release revision each time. helm rollback re-applies an earlier revision’s complete set of manifests, ConfigMap included, which is broader than kubectl rollout undo. Helm 4, released in late 2025, renamed some flags; for example --atomic, which rolls back automatically when an upgrade fails, is now --rollback-on-failure.

\n\n

You can now ship, watch and reverse a release with confidence. The last lesson gets traffic into the cluster properly and assembles the debugging playbook for when something still goes wrong.

\n","toc":[{"id":"sec-1","text":"How a rolling update works","level":2},{"id":"sec-2","text":"Deploy, watch, roll back","level":2},{"id":"sec-3","text":"Packaging with Helm","level":2}],"takeaways":["A rolling update scales a new ReplicaSet up and the old one down, paced by maxSurge and maxUnavailable and gated by readiness.","With maxUnavailable at 0, a release whose Pods never become ready stalls harmlessly while the old version keeps serving.","kubectl rollout undo switches back to the previous ReplicaSet in seconds, but update Git too or the next apply redeploys the bad version.","Rolling back code doesn’t roll back the database, so schema changes must work with both the old and the new version.","Helm packages manifests as templates plus values; helm upgrade --install deploys, and helm rollback returns to an earlier release revision."],"further":[{"title":"Deployments (rolling update and rollback)","url":"https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#rolling-back-a-deployment"},{"title":"Performing a Rolling Update","url":"https://kubernetes.io/docs/tutorials/kubernetes-basics/update/update-intro/"},{"title":"kubectl rollout","url":"https://kubernetes.io/docs/reference/kubectl/generated/kubectl_rollout/"}],"quiz":[{"q":"

notes-api runs 4 replicas with maxSurge 1 and maxUnavailable 0. Version 1.5.0’s Pods crash on start. What do users experience?

","options":[{"text":"Nothing; the rollout stalls with one failing new Pod while all 4 old Pods keep serving.","why":"Correct. With no unavailability allowed, an old Pod is only removed after a new one is ready, which never happens."},{"text":"A full outage, because the old ReplicaSet is deleted first.","why":"That describes the Recreate strategy. Rolling updates keep the old ReplicaSet until the new one is ready."},{"text":"About a quarter of requests fail until someone intervenes.","why":"That could happen with maxUnavailable above zero and no readiness probe. Here the failing Pod never receives traffic."},{"text":"Kubernetes rolls back to 1.4.0 automatically after the deadline.","why":"Kubernetes marks the rollout as failed after progressDeadlineSeconds, but it doesn’t roll back on its own."}],"answer":0},{"q":"

You ran kubectl rollout undo deployment/notes-api during an incident and service recovered. What must happen next?

","options":[{"text":"Nothing; the rollback is permanent.","why":"The cluster is fixed, but the manifest in Git still names the bad image, and the next deploy brings it back."},{"text":"Delete the old ReplicaSet so it can’t be used again.","why":"Old ReplicaSets are what make rollbacks possible. Keep them."},{"text":"Restart all Pods to clear cached state.","why":"The rollback already replaced the Pods."},{"text":"Change the image in Git (or revert the commit) so the next kubectl apply doesn’t redeploy the broken version.","why":"Correct. Git and the cluster must agree. Fix the source of truth before anyone runs the next deploy."}],"answer":3},{"q":"

Release 1.5.0 renames the database column body to content in a migration. 1.5.0 has a bug and you roll back to 1.4.0. What happens?

","options":[{"text":"The rollback also reverts the migration automatically.","why":"Kubernetes knows nothing about your database. Only the Pods change."},{"text":"1.4.0 starts failing, because it still queries body, which no longer exists.","why":"Correct. That’s why schema changes should be expand-then-contract (add new, migrate, remove old later), so adjacent versions both work."},{"text":"Postgres renames the column back when old queries arrive.","why":"Databases don’t undo schema changes in response to queries."},{"text":"Nothing breaks, because 1.4.0’s image includes its own schema.","why":"Images hold code, not database state. The schema lives in Postgres."}],"answer":1},{"q":"

You want to see the exact Kubernetes YAML a Helm chart will produce for production before installing it. Which command?

","options":[{"text":"helm history notes-api","why":"That lists past releases of an installed chart, not rendered manifests."},{"text":"helm rollback notes-api 0","why":"That changes the cluster; it doesn’t preview anything."},{"text":"helm template notes-api ./charts/notes-api -f values-prod.yaml","why":"Correct. It renders the templates with your values locally and prints the YAML, so you can read or diff it."},{"text":"kubectl get all -n notes","why":"That shows what’s already running, not what the chart would create."}],"answer":2}],"exercise":{"kind":"fill","language":"bash","template":"# Wait for the 1.5.0 rollout to finish (fails after 5 minutes)\nkubectl rollout {{0}} deployment/notes-api --timeout=5m\n\n# It's failing in production: list revisions, then go back one\nkubectl rollout {{1}} deployment/notes-api\nkubectl rollout {{2}} deployment/notes-api\n\n# On the Helm-managed staging cluster, return to release revision 4\nhelm {{3}} notes-api 4 -n notes","blanks":[{"answers":["status"]},{"answers":["history"]},{"answers":["undo"]},{"answers":["rollback"]}],"title":"Watch it, then take it back","prompt":"

Release 1.5.0 is rolling out. Fill in the blanks to wait for the rollout, list the Deployment’s revisions, roll back to the previous one, and roll the Helm-managed staging release back to revision 4.

\n","hints":["All three kubectl rollout subcommands are single words: one waits, one lists, one reverses.","Helm’s command for returning to an earlier release has the same name as the concept."],"explanation":"
Terminal
kubectl rollout status deployment/notes-api --timeout=5m\nkubectl rollout history deployment/notes-api\nkubectl rollout undo deployment/notes-api\nhelm rollback notes-api 4 -n notes
\n

status blocks until the rollout finishes and exits non-zero if it fails, which makes it a good CI gate. undo scales the previous ReplicaSet back up with the same rolling-update rules. helm rollback re-applies all manifests from release revision 4. Afterwards, revert the change in Git so the next deploy doesn’t bring 1.5.0 back.

\n","checkLabels":{}},"words":722,"readMinutes":3}