{"id":"l-dk-1-3","courseId":"docker-kubernetes","lang":"en","kind":"lesson","summary":"Order Dockerfile instructions so dependency installs stay cached, read the build cache from docker history and build output, and keep caches warm with cache mounts and CI cache exports.","html":"

A Skylane team once asked me why their CI builds had crept from forty seconds to eight minutes. Their Dockerfile looked like the one you wrote last lesson: COPY . . and then npm ci. Every commit changed some file, so every build reinstalled 900 packages from the internet. One reordering brought it back to forty seconds.

\n

Layers and cache keys

\n

Each FROM, COPY, ADD and RUN produces a layer. When you rebuild, BuildKit walks the Dockerfile top to bottom and asks of each step: “have I run exactly this before, with exactly these inputs?”

\n\n

The first step whose key doesn’t match is a cache miss, and from then on every later step rebuilds, because each one runs on top of a layer that’s now different. Cache hits are a prefix of the file. Order is everything.

\n
\n Two layer stacks. With COPY . . before npm ci, a source edit misses at COPY and reruns npm ci. With package files copied first, npm ci stays cached and only the final COPY and build rerun.\n COPY . . first\n package files first\n \n FROM node:24-slim\n \n WORKDIR /app\n \n COPY . . (src changed)\n \n RUN npm ci (~70 s)\n \n RUN npm run build\n \n FROM node:24-slim\n \n COPY package*.json\n \n RUN npm ci (cached)\n \n COPY . . (src changed)\n \n RUN npm run build\n \n cache hit\n \n rebuilt\n
One changed file invalidates its layer and every layer above it
\n

Reorder for the cache

\n

Dependencies change weekly; source changes on every commit. So copy the dependency manifests on their own, install, and only then copy the rest:

\n
Dockerfile
# syntax=docker/dockerfile:1\nFROM node:24-slim\nWORKDIR /app\n\n# Changes rarely: only when dependencies change\nCOPY package.json package-lock.json ./\nRUN npm ci\n\n# Changes on every commit\nCOPY . .\nRUN npm run build\n\nEXPOSE 3000\nCMD [\"node\", \"dist/server.js\"]
\n

Edit src/server.ts and rebuild. The output shows which steps were reused:

\n
Text
 => CACHED [2/6] WORKDIR /app\n => CACHED [3/6] COPY package.json package-lock.json ./\n => CACHED [4/6] RUN npm ci\n => [5/6] COPY . .\n => [6/6] RUN npm run build
\n

The general rule: sort instructions from least to most frequently changing. System packages, then language dependencies, then your code, then anything derived from your code.

\n

docker history notes-api:dev lists the layers with their size and the instruction that created them. Use it when an image is bigger than you expect; the culprit is usually one fat layer.

\n

The same rule applies to build arguments. A line like ARG GIT_SHA near the top, used in a later RUN or LABEL, gives every RUN step below it a new input on every commit, because build arguments are visible to every later RUN as environment variables. Declare such values as late as possible, right before the one step that uses them.

\n

COPY keys on file contents, not timestamps, so a fresh git clone on a build server hits the same cache as your laptop as long as the bytes match. That’s also why a committed, up-to-date package-lock.json matters: it is the input that decides whether the expensive install is reused.

\n

System packages: one RUN, cleaned up

\n

If you need OS packages, install them in a single RUN:

\n
Dockerfile
RUN apt-get update \\\n && apt-get install -y --no-install-recommends ca-certificates \\\n && rm -rf /var/lib/apt/lists/*
\n

--no-install-recommends skips optional extras, and deleting the package lists in the same step keeps them out of the layer. Deleting them in a later RUN doesn’t shrink anything: the earlier layer still contains them.

\n\n

Cache mounts: faster misses

\n

Sometimes the install must rerun because you added a dependency. A cache mount keeps npm’s download cache in the builder between builds, without putting it in the image:

\n
Dockerfile
COPY package.json package-lock.json ./\nRUN --mount=type=cache,target=/root/.npm npm ci
\n

npm ci still deletes and recreates node_modules, but the tarballs come from local disk instead of the registry. On notes-api that turns a 70-second reinstall into about 15.

\n

Caches in CI

\n

Your laptop keeps its cache between builds. Most CI runners start empty, so every build is cold however well you order the file. Export the cache somewhere durable and import it on the next run:

\n
Terminal
docker buildx build \\\n  --cache-from type=registry,ref=ghcr.io/skylane/notes-api:buildcache \\\n  --cache-to type=registry,ref=ghcr.io/skylane/notes-api:buildcache,mode=max \\\n  -t ghcr.io/skylane/notes-api:dev .
\n

mode=max stores intermediate layers too, which matters once you have several build stages. GitHub Actions users can use type=gha instead of a registry.

\n\n

Builds are fast now. Next you’ll get comfortable with the running side: ports, environment variables, logs, and what really happens when a container stops.

\n","toc":[{"id":"sec-1","text":"Layers and cache keys","level":2},{"id":"sec-2","text":"Reorder for the cache","level":2},{"id":"sec-3","text":"System packages: one RUN, cleaned up","level":2},{"id":"sec-4","text":"Cache mounts: faster misses","level":2},{"id":"sec-5","text":"Caches in CI","level":2}],"takeaways":["Each instruction’s cache key depends on the instruction and, for COPY, on the checksum of the files copied; one miss rebuilds every later step.","Copy package.json and package-lock.json first, run npm ci, then copy the source, so code edits don’t reinstall dependencies.","Put instructions that change rarely near the top and instructions that change on every commit near the bottom.","Run apt-get update and apt-get install in the same RUN so the package index is never stale.","A cache mount (RUN --mount=type=cache) keeps npm’s download cache between builds even when the layer itself must rebuild."],"further":[{"title":"Docker build cache","url":"https://docs.docker.com/build/cache/"},{"title":"Optimize cache usage in builds","url":"https://docs.docker.com/build/cache/optimize/"},{"title":"Cache storage backends","url":"https://docs.docker.com/build/cache/backends/"}],"quiz":[{"q":"

In this Dockerfile you edit only src/routes.ts and rebuild. Which steps run again?

\n
Dockerfile
FROM node:24-slim\nWORKDIR /app\nCOPY package.json package-lock.json ./\nRUN npm ci\nCOPY . .\nRUN npm run build
\n","options":[{"text":"Every step, because the build context changed.","why":"The context changing doesn’t invalidate everything. Each step is checked on its own until the first miss."},{"text":"Only RUN npm run build, because that’s the step that compiles TypeScript.","why":"COPY . . copies src/routes.ts, so its checksum changes and it misses first; the build step then reruns because it comes after a miss."},{"text":"RUN npm ci and everything after it.","why":"npm ci sits after a COPY of only the two package files, which didn’t change, so it’s still a cache hit."},{"text":"COPY . . and RUN npm run build; the dependency install stays cached.","why":"Correct. The first changed input is at COPY . ., so it and every later step rerun. The expensive install above it is reused."}],"answer":3},{"q":"

Why does RUN apt-get update on its own line, followed by RUN apt-get install -y curl, cause trouble months later?

","options":[{"text":"Docker refuses to cache apt-get commands.","why":"Docker caches RUN steps by their text, apt or not. That’s exactly the problem here."},{"text":"The update layer stays cached with an old package index, so a later change to the install line fetches stale or missing versions.","why":"Correct. The cache key is the command text, which never changes, so the index is from the day the cache was created. Combine them in one RUN."},{"text":"Two RUN instructions double the image size.","why":"Extra layers add a little metadata, not double the size. The real issue is a stale cached index."},{"text":"apt-get update needs root and the second RUN drops privileges.","why":"Both run as the same user unless a USER instruction comes between them."}],"answer":1},{"q":"

Your CI runner starts with an empty Docker cache on every job, so every build is cold. What helps most?

","options":[{"text":"Add more RUN instructions so there are more layers to cache.","why":"More layers don’t help when there’s no cache to read from in the first place."},{"text":"Switch from npm ci to npm install because it’s faster.","why":"npm install may rewrite the lockfile and doesn’t fix the empty cache. Keep npm ci for reproducibility."},{"text":"Export and import the build cache, for example with --cache-to and --cache-from pointing at a registry.","why":"Correct. Cache backends let fresh runners reuse layers built by earlier jobs."},{"text":"Use docker build --no-cache so the result is predictable.","why":"That throws away caching entirely, which is the opposite of what you want for speed."}],"answer":2},{"q":"

What does RUN --mount=type=cache,target=/root/.npm npm ci give you that layer caching doesn’t?

","options":[{"text":"npm’s download cache persists between builds, so even when this step must rerun, packages come from local disk instead of the network.","why":"Correct. A layer cache is all-or-nothing; a cache mount survives a layer miss and isn’t stored in the image."},{"text":"The node_modules folder is mounted from your laptop into the image.","why":"Cache mounts live in the builder, not on your host, and they cache npm’s download store, not node_modules."},{"text":"The installed packages are excluded from the final image to save space.","why":"node_modules is still written into the layer. Only the mounted /root/.npm directory stays out of the image."},{"text":"The step is never rerun, even if the lockfile changes.","why":"The step reruns whenever its inputs change. The mount only makes the rerun faster."}],"answer":0}],"exercise":{"kind":"spot-bug","language":"dockerfile","code":"# syntax=docker/dockerfile:1\nFROM node:24-slim\nRUN apt-get update\nRUN apt-get install -y --no-install-recommends ca-certificates\nWORKDIR /app\nCOPY . .\nRUN npm ci\nRUN npm run build\nEXPOSE 3000\nCMD [\"node\", \"dist/server.js\"]","bugLines":[3,6],"title":"Find the cache killers","prompt":"

CI builds for this Dockerfile take eight minutes even when a commit changes one line of TypeScript, and last week adding a second OS package failed with Unable to locate package. Select the two lines responsible.

\n","hints":["Which line’s cache key changes on every commit, and what runs after it?","Which RUN is cached by its text alone, so its result can be months old?"],"explanation":"

Line 6 copies the whole context before npm ci, so any source change misses the cache there and reinstalls every dependency. Copy only the manifests first, install, then copy the rest.

\n

Line 3 caches apt-get update on its own; when the install line changes, it runs against an index from the day the cache was made. Chain update and install in one RUN:

\n
Dockerfile
RUN apt-get update \\\n && apt-get install -y --no-install-recommends ca-certificates \\\n && rm -rf /var/lib/apt/lists/*\nWORKDIR /app\nCOPY package.json package-lock.json ./\nRUN npm ci\nCOPY . .\nRUN npm run build
\n","checkLabels":{}},"words":678,"readMinutes":3}