The Allowlist That Isn't: Substring Matching in an LLM Tool Guard

AI Security Level 3/4 ~5 min September 18, 2026

The challenge

Atlas is the assistant Halcyon runs for its own staff. It has exactly one tool that touches the network: fetch_url, so it can read a page a colleague links to before answering questions about it. The team knew that handing a model an HTTP client is how you end up with server-side request forgery, so they wrote a guard before they shipped it. Only three documentation domains are reachable. Only http and https. A denylist covers the cloud metadata address and loopback. It passed review twice. It still does not hold. One of the three checks accepts hosts that nobody at Halcyon owns, and anyone who can get a link in front of Atlas can point the tool at infrastructure they control. Read the three files and name the function whose check lets attacker-owned hosts through.

What you'll learn

  • Tell a membership test apart from a substring test in Python
  • Explain why substring matching is safe in a denylist and unsafe in an allowlist
  • Construct a hostname that defeats a substring allowlist
  • Describe how an agent tool turns SSRF into readable output for the attacker
  • Write a host comparison that cannot be widened by an attacker-owned suffix

Skills tested

Source code reviewSSRF reasoningAI agent tool security

Prerequisites

  • Reading Python functions and comprehensions
  • What a hostname and a subdomain are

How it works

Giving a model an HTTP client is giving it a request forger. The model decides the URL, the server makes the request, and the server sits inside your network. So the whole security property of a fetch_url tool lives in one question: is this host one we chose to allow?

Atlas answers that question in host_allowed, and it answers a different one by accident. any(domain in host for domain in ALLOWED_DOMAINS) looks like a membership test, and in scheme_allowed the same keyword really is one, because urlparse(url).scheme in ("http", "https") compares against the elements of a tuple. Between two strings, in means substring. host_allowed is asking whether the text docs.halcyon.example appears somewhere inside the host.

An attacker who owns any domain at all owns an infinite number of hosts that contain that text. docs.halcyon.example.attacker.example is an ordinary subdomain of attacker.example. It costs nothing, it resolves wherever they point it, and it satisfies the check. Burying the allowed name in the middle works equally well.

The denylist a few lines above is also a substring test, and that one is not a bug. Widening a denylist makes it catch more, so any(bad in url.lower() for bad in BLOCKED) refuses more URLs than intended rather than fewer. Direction matters: loose matching is dangerous when it grants and merely clumsy when it denies.

What makes this worse than a blind SSRF is that the agent reports back. The response body is appended to the conversation as tool output and then summarised to whoever is talking to Atlas, so the attacker does not need a side channel to read what the server fetched.

Common mistakes

  • Answering scheme_allowed. It uses the same keyword but compares against a tuple, so it is a real membership test and it is correct.
  • Answering target_allowed. Its ordering and coverage are fine; it fails only because the helper it trusts is wrong.
  • Calling the denylist the bug. Substring matching in a denylist over-blocks, which is the safe direction.
  • Blaming allow_redirects. It is explicitly set to False, which is the one thing the authors got right for the right reason.
  • Thinking the metadata address is required. Any attacker host is already enough, because the fetched body comes back to the model.

How to defend against it

Compare hosts the way DNS does: as whole labels, with equality.

  • Parse once with urlparse, take hostname rather than netloc so credentials and ports are already stripped, lowercase it and drop a trailing dot.
  • Test it against a set with == or in on that set. If subdomains must be allowed, check host == d or host.endswith("." + d), never a bare substring.
  • Resolve the hostname and reject the request if the address is private, loopback, link-local or multicast, then connect to the address you validated so DNS cannot change under you.
  • Keep redirects off, or re-run the full guard on every hop.
  • Egress-filter the container the agent runs in. The allowlist is the first control, not the only one.

Full solution

Pro and Max members unlock the complete step-by-step walkthrough.

Go Pro

Community stats

125 completions
75% success rate
M2F14M3 First blood

Related Daily Hacks

Contributors

Credited contributors

These hackers reported issues, improved content, and helped harden this page.

27,000+ Hackers 100+ Labs & Courses Free
Start Hacking Free