Find the Tool-Exec Sink: RCE Through an LLM Agent

Segurança de IA Nível 3/4 ~5 min 2026-08-08

O desafio

Este agente LLM deixa o modelo chamar ferramentas retornando JSON. Uma ferramenta executa o código que o modelo coloca na resposta, sem nenhum sandbox - então uma saída de modelo envenenada vira execução remota de código. Leia os dois arquivos e digite a chamada que executa esse código.

Faça login para tentar

O que você vai aprender

Recognise exec()/eval() on LLM output as a remote code execution sink
Trace data from a model reply through JSON parsing to a tool dispatcher
Understand how prompt injection escalates into code execution
Tell apart executing code (exec) from safely parsing data (ast.literal_eval)
Explain why model output must be treated as untrusted input

Habilidades testadas

AI agent code reviewPython injection sinksPrompt-injection threat modelling

Pré-requisitos

Basic Python reading
How an LLM tool/function call works at a high level
What prompt injection is

Como funciona

Tool-calling agents let a model act: it returns a structured request (a tool name and arguments) and the runtime executes it. The security problem is that model output is not trusted data. An attacker can steer it through prompt injection - a malicious instruction hidden in a user message, a web page, or a retrieved document - so any tool that runs raw model output is an injection sink.

In agent.py, run_tool reads the tool name from the model's JSON and, for the python tool, calls exec(args['code']). Whatever string the model places in args['code'] is executed as Python on the host, with no sandbox and full process privileges. A poisoned prompt can make the model emit {"tool": "python", "args": {"code": "__import__('os').system('...')"}} and the agent runs it - this is remote code execution that originates from text, not a network port.

The contrast is in tools.py. parse_number uses ast.literal_eval, which converts a string into a number, list, or dict and refuses anything executable. That is the right primitive when you need data from text. The reviewer's job is to follow the model reply to the call that runs it - and the call that runs it is exec.

Erros comuns

Pointing at json.loads. Parsing the reply is fine; the bug is executing one of its fields. Name the call that runs the code.
Picking ast.literal_eval. That is the safe helper in tools.py - it parses data and never executes.
Assuming the system prompt protects you. A model can be talked out of any instruction; the runtime must not trust its output.
Thinking 'it is just a calculator tool'. The python tool runs arbitrary code, not arithmetic - the feature name hides the sink.

Como se proteger

Never pass model output to exec() or eval(). Treat every tool call from the model as attacker-controlled and constrain what tools can do.

Replace a raw code tool with a fixed allow-list of typed, parameterised tools (no free-form code).
If code execution is truly required, run it in a strong sandbox (separate container, no network, dropped privileges, resource limits).
Validate and schema-check tool arguments before dispatch; reject anything unexpected.
Use ast.literal_eval or json.loads when you only need to turn text into data.