{"id":"l-dk-5-2","courseId":"docker-kubernetes","lang":"en","kind":"lesson","summary":"Set CPU and memory requests and limits for notes-api from measurements, predict throttling, OOM kills and QoS classes, and scale replicas automatically with an autoscaling/v2 HorizontalPodAutoscaler.","html":"

The first time notes-api met real traffic on a shared cluster, two things happened within an hour. A batch job on the same node used all the memory and one notes-api Pod was evicted. Then a traffic spike pegged the CPU, latency went to seconds, and there were still exactly three Pods because nobody had told Kubernetes it could add more. Both failures came from the same gap: Kubernetes didn’t know what notes-api needed.

\n

Requests and limits

\n

Every container can declare two numbers per resource:

\n
k8s/deployment.yaml
        - name: api\n          image: ghcr.io/skylane/notes-api:1.4.0\n          resources:\n            requests:\n              cpu: 250m\n              memory: 256Mi\n            limits:\n              memory: 512Mi
\n\n

The two resources behave very differently when a container hits its limit:

\n\n

If you set a limit without a request, Kubernetes copies the limit into the request. A request larger than its limit is rejected outright.

\n\n

For Node.js, give V8 a heap ceiling below the container’s memory limit, for example NODE_OPTIONS=--max-old-space-size=384 with a 512Mi limit. Then the garbage collector works hard as memory fills, instead of the kernel ending the process without warning.

\n

Choosing the numbers

\n

Don’t guess. Run notes-api under realistic load and measure:

\n
Terminal
kubectl top pods -l app.kubernetes.io/name=notes-api
\n

kubectl top needs metrics-server, which many managed clusters include or offer as an add-on; on kind you install it yourself. Set the CPU request near typical usage under normal load, and the memory request and limit with headroom above the peak you observed. Revisit the numbers after major releases. Requests that are too high waste money on idle reservations; too low, and Pods are packed onto nodes that can’t actually carry them.

\n

QoS classes and eviction

\n

From requests and limits, Kubernetes derives a Quality of Service class for each Pod:

\n
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
ClassRuleEvicted under node memory pressure
GuaranteedEvery container has CPU and memory requests equal to limitsLast
BurstableAt least one request or limit set, but not GuaranteedAfter BestEffort, starting with Pods furthest over their requests
BestEffortNo requests or limits anywhereFirst

notes-api above is Burstable. The batch job that caused our eviction was BestEffort, so with requests in place it would have been evicted before notes-api. Running production workloads as BestEffort means volunteering them to go first.

\n

Horizontal Pod Autoscaling

\n

The HorizontalPodAutoscaler (HPA) adjusts a Deployment’s replica count from metrics. It’s another control loop: every 15 seconds or so it compares observed usage with a target and computes the replica count that would bring usage back to the target.

\n
k8s/hpa.yaml
apiVersion: autoscaling/v2\nkind: HorizontalPodAutoscaler\nmetadata:\n  name: notes-api\n  namespace: notes\nspec:\n  scaleTargetRef:\n    apiVersion: apps/v1\n    kind: Deployment\n    name: notes-api\n  minReplicas: 3\n  maxReplicas: 12\n  metrics:\n    - type: Resource\n      resource:\n        name: cpu\n        target:\n          type: Utilization\n          averageUtilization: 70\n  behavior:\n    scaleDown:\n      stabilizationWindowSeconds: 300
\n

averageUtilization: 70 means “keep average CPU usage at 70% of the CPU request”. With a 250m request, that’s about 175m per Pod. At six Pods averaging 350m, the HPA computes 6 × 350 / 175 = 12 and scales to the maximum. The behavior block makes scale-down wait for five minutes of consistently lower load, so a brief lull doesn’t remove capacity you need a minute later. Five minutes is already the default; writing it down makes the choice visible and easy to tune.

\n
\n metrics-server collects CPU usage from the Pods. The HPA reads it, compares usage with 70 percent of requests, and updates the Deployment's replicas, which changes the number of Pods.\n \n metrics-server\n \n HPA\n target 70% of request\n \n Deployment\n replicas: 3 → 6\n \n \n \n notes-api Pods\n \n \n CPU usage\n every ~15 s\n
The HPA is a control loop over the Deployment’s replica count
\n\n

Watch it work with kubectl get hpa notes-api --watch while you run a load test. If the targets column shows <unknown>, either the containers have no CPU request or metrics-server isn’t running.

\n

CPU suits notes-api because its work is CPU-bound request handling. Services whose load shows up elsewhere, like queue depth or requests per second, can scale on custom or external metrics through the same metrics list, with an adapter installed in the cluster.

\n

notes-api now has the resources it needs and grows with load. The next lesson changes the image safely: rolling updates, rollbacks and packaging the whole thing with Helm.

\n","toc":[{"id":"sec-1","text":"Requests and limits","level":2},{"id":"sec-2","text":"Choosing the numbers","level":2},{"id":"sec-3","text":"QoS classes and eviction","level":2},{"id":"sec-4","text":"Horizontal Pod Autoscaling","level":2}],"takeaways":["Requests are what the scheduler reserves for a container; limits are the ceiling enforced at run time.","Going over a CPU limit throttles the container; going over a memory limit gets it OOM-killed with exit code 137.","QoS class follows from requests and limits: Guaranteed (equal for every container), Burstable (some set), BestEffort (none), and BestEffort Pods are evicted first.","An HPA’s CPU utilization target is a percentage of the CPU request, so it can’t work on containers without one.","When an HPA manages a Deployment, remove replicas from the Deployment manifest so kubectl apply doesn’t fight the autoscaler."],"further":[{"title":"Resource Management for Pods and Containers","url":"https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/"},{"title":"Pod Quality of Service Classes","url":"https://kubernetes.io/docs/concepts/workloads/pods/pod-qos/"},{"title":"Horizontal Pod Autoscaling","url":"https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/"}],"quiz":[{"q":"

A notes-api Pod restarts several times a day. kubectl describe pod shows Reason: OOMKilled and Exit Code: 137. What happened?

","options":[{"text":"The CPU limit was too low, so the kernel killed the container.","why":"CPU over the limit is throttled (slowed down), never killed."},{"text":"The liveness probe failed and Kubernetes restarted the container.","why":"A liveness restart shows a probe failure in the events, not OOMKilled as the reason."},{"text":"The container used more memory than its limit, and the kernel’s OOM killer ended it.","why":"Correct. Raise the memory limit to fit real usage, or find the leak; for Node.js, also cap the V8 heap below the limit."},{"text":"The node ran out of disk space for logs.","why":"Disk pressure leads to eviction with a different reason, not an OOM kill."}],"answer":2},{"q":"

An HPA targets 70% CPU utilization, and kubectl get hpa shows cpu: <unknown>/70%. What’s the most likely cause?

","options":[{"text":"The HPA uses autoscaling/v2, which doesn’t support CPU.","why":"autoscaling/v2 supports CPU, memory, custom and external metrics."},{"text":"The notes-api containers have no CPU request, so utilization can’t be computed (or metrics-server isn’t installed).","why":"Correct. Utilization is usage divided by request. No request, no percentage. Also check that metrics-server is running."},{"text":"minReplicas is higher than the current replica count.","why":"That makes the HPA scale up to the minimum; it doesn’t make metrics unknown."},{"text":"The Deployment has too many replicas for the HPA to manage.","why":"There’s no such limit; the HPA can manage any number within its bounds."}],"answer":1},{"q":"

notes-api sets requests: {cpu: 250m, memory: 256Mi} and limits: {memory: 512Mi}. Which QoS class does the Pod get?

","options":[{"text":"Guaranteed","why":"Guaranteed needs requests equal to limits for both CPU and memory in every container."},{"text":"BestEffort","why":"BestEffort means no requests or limits at all. This Pod has requests."},{"text":"Critical","why":"That’s not a QoS class. The three classes are Guaranteed, Burstable and BestEffort."},{"text":"Burstable","why":"Correct. It has requests, but they don’t equal limits for every resource, so it can burst above its requests."}],"answer":3},{"q":"

notes-api’s Deployment manifest says replicas: 3, and an HPA has scaled it to 9 during a traffic peak. What happens when CI runs kubectl apply -f k8s/?

","options":[{"text":"The Deployment drops to 3 replicas until the HPA scales it back up, causing a capacity dip in the middle of the peak.","why":"Correct. Apply sets the field from the file. Remove replicas from the manifest once an HPA owns it."},{"text":"Nothing; the HPA locks the replicas field.","why":"The HPA writes the field, but it doesn’t lock it. Anything else that writes it wins until the next HPA decision."},{"text":"Apply fails with a conflict error.","why":"With ordinary client-side apply there’s no error; it overwrites silently, which is what makes this dangerous."},{"text":"The HPA’s minReplicas changes to 3.","why":"Applying a Deployment never edits the HPA object."}],"answer":0}],"exercise":{"kind":"spot-bug","language":"yaml","code":"apiVersion: apps/v1\nkind: Deployment\nmetadata:\n name: notes-api\nspec:\n replicas: 3\n # …selector and template labels as before\n template:\n spec:\n containers:\n - name: api\n image: ghcr.io/skylane/notes-api:1.4.0\n resources:\n limits:\n memory: 512Mi\n---\napiVersion: autoscaling/v2\nkind: HorizontalPodAutoscaler\nmetadata:\n name: notes-api\nspec:\n scaleTargetRef:\n apiVersion: apps/v1\n kind: Deployment\n name: notes-api\n minReplicas: 3\n maxReplicas: 12\n metrics:\n - type: Resource\n resource:\n name: cpu\n target:\n type: Utilization\n averageUtilization: 70","bugLines":[6,14],"title":"The autoscaler that never scales","prompt":"

kubectl get hpa notes-api shows cpu: <unknown>/70% even though metrics-server is running, and on busy days the replica count drops to 3 every time CI deploys. Select the two lines responsible.

\n","hints":["CPU utilization is measured as a percentage of something. Is that something defined for this container?","Which field do both the HPA and every kubectl apply write to?"],"explanation":"

Line 14: the container sets only a memory limit, so there’s no CPU request, and the HPA can’t compute utilization as a percentage of it. Add requests:

\n
YAML
          resources:\n            requests:\n              cpu: 250m\n              memory: 256Mi\n            limits:\n              memory: 512Mi
\n

Line 6: replicas: 3 in the manifest means every kubectl apply resets the count, fighting the HPA. Remove the field once an HPA manages the Deployment; minReplicas now holds the floor.

\n","checkLabels":{}},"words":826,"readMinutes":4}