The Allowlist That Isn't: Substring Matching in an LLM Tool Guard
The challenge
Atlas is the assistant Halcyon runs for its own staff. It has exactly one tool that touches the network: fetch_url, so it can read a page a colleague links to before answering questions about it. The team knew that handing a model an HTTP client is how you end up with server-side request forgery, so they wrote a guard before they shipped it. Only three documentation domains are reachable. Only http and https. A denylist covers the cloud metadata address and loopback. It passed review twice. It still does not hold. One of the three checks accepts hosts that nobody at Halcyon owns, and anyone who can get a link in front of Atlas can point the tool at infrastructure they control. Read the three files and name the function whose check lets attacker-owned hosts through.
What you'll learn
- Tell a membership test apart from a substring test in Python
- Explain why substring matching is safe in a denylist and unsafe in an allowlist
- Construct a hostname that defeats a substring allowlist
- Describe how an agent tool turns SSRF into readable output for the attacker
- Write a host comparison that cannot be widened by an attacker-owned suffix
Skills tested
Prerequisites
- Reading Python functions and comprehensions
- What a hostname and a subdomain are
How it works
Giving a model an HTTP client is giving it a request forger. The model decides the URL, the server makes the request, and the server sits inside your network. So the whole security property of a fetch_url tool lives in one question: is this host one we chose to allow?
Atlas answers that question in host_allowed, and it answers a different one by accident. any(domain in host for domain in ALLOWED_DOMAINS) looks like a membership test, and in scheme_allowed the same keyword really is one, because urlparse(url).scheme in ("http", "https") compares against the elements of a tuple. Between two strings, in means substring. host_allowed is asking whether the text docs.halcyon.example appears somewhere inside the host.
An attacker who owns any domain at all owns an infinite number of hosts that contain that text. docs.halcyon.example.attacker.example is an ordinary subdomain of attacker.example. It costs nothing, it resolves wherever they point it, and it satisfies the check. Burying the allowed name in the middle works equally well.
The denylist a few lines above is also a substring test, and that one is not a bug. Widening a denylist makes it catch more, so any(bad in url.lower() for bad in BLOCKED) refuses more URLs than intended rather than fewer. Direction matters: loose matching is dangerous when it grants and merely clumsy when it denies.
What makes this worse than a blind SSRF is that the agent reports back. The response body is appended to the conversation as tool output and then summarised to whoever is talking to Atlas, so the attacker does not need a side channel to read what the server fetched.
Common mistakes
- Answering scheme_allowed. It uses the same keyword but compares against a tuple, so it is a real membership test and it is correct.
- Answering target_allowed. Its ordering and coverage are fine; it fails only because the helper it trusts is wrong.
- Calling the denylist the bug. Substring matching in a denylist over-blocks, which is the safe direction.
- Blaming allow_redirects. It is explicitly set to False, which is the one thing the authors got right for the right reason.
- Thinking the metadata address is required. Any attacker host is already enough, because the fetched body comes back to the model.
How to defend against it
Compare hosts the way DNS does: as whole labels, with equality.
- Parse once with
urlparse, takehostnamerather thannetlocso credentials and ports are already stripped, lowercase it and drop a trailing dot. - Test it against a
setwith==orinon that set. If subdomains must be allowed, checkhost == d or host.endswith("." + d), never a bare substring. - Resolve the hostname and reject the request if the address is private, loopback, link-local or multicast, then connect to the address you validated so DNS cannot change under you.
- Keep redirects off, or re-run the full guard on every hop.
- Egress-filter the container the agent runs in. The allowlist is the first control, not the only one.
Full solution
Community stats
Related Daily Hacks
Credited contributors
These hackers reported issues, improved content, and helped harden this page.