Client-Controlled Debug Mode Leaks the Hidden System Prompt
The challenge
This is the request the chat widget sends to the AI backend. The client gets to set a debug flag in the body, and the server honors it - when debug is on, the response echoes back the hidden system prompt the model was given. Turn debug on, send it, and read the flag baked into that leaked prompt.
What you'll learn
- Recognize a debug-mode information-disclosure flaw on an AI chat endpoint
- Spot a debug flag in a request body that the client should never control
- Tamper a boolean field in a captured request to reveal hidden server state
- Read an API response to extract a leaked system prompt and the secret inside it
- Explain why debug output and system prompts must be gated server-side and never returned to clients
Skills tested
Prerequisites
- Basic understanding of HTTP requests, JSON bodies, and API authentication headers
- Familiarity with the idea that an LLM is steered by a hidden system prompt
How it works
Large-language-model applications are steered by a system prompt - a block of instructions, persona, and sometimes secrets that the operator gives the model before the user's message. That prompt is server-side configuration. It can contain internal tool names, policy rules, and, as here, build or integration tokens, so it must never be returned to the client.
This endpoint shipped a debug feature that, when enabled, attaches the raw model context - including the system prompt - to the response. The mistake is that the flag controlling it lives in the request body, so any caller can set debug=true and receive internal state. The value is a valid boolean; nothing is malformed. The flaw is that a development convenience was wired to a client-controlled switch and left reachable in production.
This is a classic information disclosure through a leftover debug path, and it is especially dangerous for AI systems because the leaked system prompt reveals how to manipulate the model and may carry credentials. Debug output must be gated by trusted server-side signals - an environment flag, an internal network, an operator role - never by a parameter the caller can flip.
Common mistakes
- Assuming the debug flag is purely cosmetic or read-only, when the server actually changes its response based on it.
- Trying to jailbreak the model with a clever message instead of noticing the debug switch that hands you the prompt directly.
- Sending a non-boolean such as
1oronand giving up when the server rejects it - the field is validated as a boolean, so it must betrue. - Treating the leak as the model misbehaving rather than the real flaw: the server attached internal state to a response because a client-controlled flag told it to.
How to defend against it
Never let a client decide whether debug or verbose internal output is returned, and never include the system prompt or other server-side configuration in any client-facing response. Gate any diagnostic output behind trusted server-side signals.
- Drive debug behavior from a server environment variable or an authenticated operator role, and ignore any
debugfield sent by the client. - Strip internal context - system prompt, model name, tool definitions, tokens - from responses before they leave the server.
- Keep secrets out of the system prompt entirely; inject credentials at the tool layer where the model never sees them.
- Log and alert when a request attempts to enable a debug or verbose mode, so probing is visible.