Deleted in the Last Layer: Recovering a Secret From a Docker Image
The challenge
Pinewood's last build step deleted its .env. Image layers never forget: dig through ~/image. Recover the STRIPE_KEY that .env held.
What you'll learn
- Explain how a Docker image is built from stacked layers, in manifest order
- Recognise a whiteout file and what it hides
- Locate a deleted file in an earlier layer and read it
- Keep build secrets out of image layers
Skills tested
Container forensicsLinux file searchSecret hunting
Prerequisites
- Basic shell commands (ls, cat, find)
- What a Dockerfile is
How it works
Every COPY, ADD and RUN in a Dockerfile produces a new read-only layer. The image is the stack of all of them, and a container sees the merged view.
When a later step deletes a file, Docker cannot rewrite the earlier layer. It adds a whiteout entry instead, a file named .wh.<name>, which tells the merged view to hide the original. The original bytes are still shipped inside the earlier layer.
docker save exports every layer as a tarball, so anyone who can pull a public image can read every file any build step ever wrote.
Common mistakes
- Using plain
ls..envis a dotfile, so it only shows withls -aorfind. - Reading the whiteout.
.wh..envis empty: it is the delete marker, not the file. - Taking the first .env you find. Two layers wrote one. Layer ids do not sort in build order:
manifest.jsondoes, and the later write is the one production used. - Taking the example file.
.env.exampleholds a placeholdersk_test_value.
How to defend against it
- Exclude
.envand other secret files from the build context in.dockerignore. - Pass build-time secrets with BuildKit:
RUN --mount=type=secret,id=env ...mounts the secret for one step and never writes it to a layer. - Use multi-stage builds so only the final artefacts reach the shipped image.
- Inject runtime secrets as environment variables or from a secrets manager, and rotate any key that was ever in a pushed image.