Find the Tool-Exec Sink: RCE Through an LLM Agent

Sécurité de l'IA Niveau 3/4 ~5 min 2026-08-08

Le défi

Cet agent LLM laisse le modèle appeler des outils en renvoyant du JSON. Un outil exécute le code que le modèle place dans sa réponse, sans aucun bac à sable - une sortie de modèle empoisonnée devient donc une exécution de code à distance. Lisez les deux fichiers et tapez l'appel qui exécute ce code.

Connecte-toi pour essayer

Ce que tu vas apprendre

Recognise exec()/eval() on LLM output as a remote code execution sink
Trace data from a model reply through JSON parsing to a tool dispatcher
Understand how prompt injection escalates into code execution
Tell apart executing code (exec) from safely parsing data (ast.literal_eval)
Explain why model output must be treated as untrusted input

Compétences testées

AI agent code reviewPython injection sinksPrompt-injection threat modelling

Prérequis

Basic Python reading
How an LLM tool/function call works at a high level
What prompt injection is

Comment ça marche

Tool-calling agents let a model act: it returns a structured request (a tool name and arguments) and the runtime executes it. The security problem is that model output is not trusted data. An attacker can steer it through prompt injection - a malicious instruction hidden in a user message, a web page, or a retrieved document - so any tool that runs raw model output is an injection sink.

In agent.py, run_tool reads the tool name from the model's JSON and, for the python tool, calls exec(args['code']). Whatever string the model places in args['code'] is executed as Python on the host, with no sandbox and full process privileges. A poisoned prompt can make the model emit {"tool": "python", "args": {"code": "__import__('os').system('...')"}} and the agent runs it - this is remote code execution that originates from text, not a network port.

The contrast is in tools.py. parse_number uses ast.literal_eval, which converts a string into a number, list, or dict and refuses anything executable. That is the right primitive when you need data from text. The reviewer's job is to follow the model reply to the call that runs it - and the call that runs it is exec.

Erreurs fréquentes

Pointing at json.loads. Parsing the reply is fine; the bug is executing one of its fields. Name the call that runs the code.
Picking ast.literal_eval. That is the safe helper in tools.py - it parses data and never executes.
Assuming the system prompt protects you. A model can be talked out of any instruction; the runtime must not trust its output.
Thinking 'it is just a calculator tool'. The python tool runs arbitrary code, not arithmetic - the feature name hides the sink.

Comment s'en protéger

Never pass model output to exec() or eval(). Treat every tool call from the model as attacker-controlled and constrain what tools can do.

Replace a raw code tool with a fixed allow-list of typed, parameterised tools (no free-form code).
If code execution is truly required, run it in a strong sandbox (separate container, no network, dropped privileges, resource limits).
Validate and schema-check tool arguments before dispatch; reject anything unexpected.
Use ast.literal_eval or json.loads when you only need to turn text into data.