Reading a Secret Leaked Inside an LLM System Prompt
The challenge
You have a shell on a machine that runs a company chatbot. The operator wrote a secret directly into the model's system prompt, thinking users would never see it. Find the system prompt file on disk, read it, and submit the secret embedded inside.
What you'll learn
- Locate where an inference server loads its system prompt from
- Follow a config path to the system prompt file on disk
- Read a secret an operator embedded in the prompt text
- Use grep to find a flagged value across the model directory
- Explain why a system prompt cannot hold secrets safely
Skills tested
Prerequisites
- Comfortable with basic shell commands (ls, cat, grep)
- Know that an LLM is steered by a system prompt
- Understand that a config file can point to other files
How it works
A system prompt is the standing instruction an application prepends to every conversation with a language model - tone, rules, and context the model should follow. Because the end user never types it and the chat UI never shows it, operators sometimes treat it as a private channel and tuck secrets inside: internal flags, API keys, escalation codes. That assumption is the vulnerability this challenge exercises.
The system prompt has to come from somewhere, and on a real deployment it is almost always a file the server reads at start-up. Here config.json spells out system_prompt_path: /opt/model/system_prompt.txt, and serve.py opens exactly that path. Anyone with read access to the inference host can cat the file and see every word the operator thought was hidden - including the line The internal escalation flag is HDNA{prompts_are_not_secrets}.
Even without a shell, a system prompt is leaky. Models can be coaxed into repeating their instructions through prompt-injection or simple social-engineering style requests, so a secret in the prompt is one clever message away from any user of the chatbot. The lesson is blunt: a system prompt steers behaviour, it does not keep secrets. Anything placed in it should be treated as readable by both the host's operators and, in practice, the model's users.
Common mistakes
- Treating the system prompt as private. Users never type it, but it is a plain file on disk and can be coaxed out of the model - it hides nothing.
- Opening the weights instead of the prompt. The flag is text in
system_prompt.txt, not in the binarymodel.safetensors. - Missing the config pointer.
config.jsonnamessystem_prompt_path; ignoring it means guessing where the prompt lives. - Reading past the marked line. The operator labelled the value as the internal escalation flag - that line is the answer, not the support hours next to it.
How to defend against it
Secrets must never live in a prompt. The fixes keep credentials out of the model's context entirely:
- Store real secrets in a secret manager or environment variable and reference them from application code, never inside the system prompt text.
- Have the application enforce sensitive actions server-side - the model should ask the backend to check staff identity, not hold an escalation flag it might reveal.
- Restrict read access to the inference host and the prompt file so a foothold does not immediately expose the prompt.
- Add prompt-injection testing to release checks so the model is verified not to leak its instructions on demand.
If a secret has been placed in a prompt, rotate it and remove it from the prompt - assume it has already been read by both anyone on the host and any user who asked the model the right way.