Client-Controlled Debug Mode Leaks the Hidden System Prompt
O desafio
Esta é a requisição que o widget de chat envia ao backend de IA. O cliente pode definir uma flag debug no corpo, e o servidor a respeita - quando debug está ligado, a resposta devolve o prompt de sistema oculto dado ao modelo. Ligue o debug, envie e leia o flag embutido nesse prompt vazado.
O que você vai aprender
- Recognize a debug-mode information-disclosure flaw on an AI chat endpoint
- Spot a debug flag in a request body that the client should never control
- Tamper a boolean field in a captured request to reveal hidden server state
- Read an API response to extract a leaked system prompt and the secret inside it
- Explain why debug output and system prompts must be gated server-side and never returned to clients
Habilidades testadas
Pré-requisitos
- Basic understanding of HTTP requests, JSON bodies, and API authentication headers
- Familiarity with the idea that an LLM is steered by a hidden system prompt
Como funciona
Large-language-model applications are steered by a system prompt - a block of instructions, persona, and sometimes secrets that the operator gives the model before the user's message. That prompt is server-side configuration. It can contain internal tool names, policy rules, and, as here, build or integration tokens, so it must never be returned to the client.
This endpoint shipped a debug feature that, when enabled, attaches the raw model context - including the system prompt - to the response. The mistake is that the flag controlling it lives in the request body, so any caller can set debug=true and receive internal state. The value is a valid boolean; nothing is malformed. The flaw is that a development convenience was wired to a client-controlled switch and left reachable in production.
This is a classic information disclosure through a leftover debug path, and it is especially dangerous for AI systems because the leaked system prompt reveals how to manipulate the model and may carry credentials. Debug output must be gated by trusted server-side signals - an environment flag, an internal network, an operator role - never by a parameter the caller can flip.
Erros comuns
- Assuming the debug flag is purely cosmetic or read-only, when the server actually changes its response based on it.
- Trying to jailbreak the model with a clever message instead of noticing the debug switch that hands you the prompt directly.
- Sending a non-boolean such as
1oronand giving up when the server rejects it - the field is validated as a boolean, so it must betrue. - Treating the leak as the model misbehaving rather than the real flaw: the server attached internal state to a response because a client-controlled flag told it to.
Como se proteger
Never let a client decide whether debug or verbose internal output is returned, and never include the system prompt or other server-side configuration in any client-facing response. Gate any diagnostic output behind trusted server-side signals.
- Drive debug behavior from a server environment variable or an authenticated operator role, and ignore any
debugfield sent by the client. - Strip internal context - system prompt, model name, tool definitions, tokens - from responses before they leave the server.
- Keep secrets out of the system prompt entirely; inject credentials at the tool layer where the model never sees them.
- Log and alert when a request attempts to enable a debug or verbose mode, so probing is visible.