Who Pulled the Keys: Finding Key Harvesting in App Logs
The challenge
A debug endpoint was left exposed and it serves private key material. In your app logs, most traffic is normal app pages, but one IP keeps hitting /debug/keys and /.well-known/private-keys and getting a 200 every time - it is quietly downloading the secrets. Filter the suspicious paths, find the single source pulling key material, and submit its IP.
What you'll learn
- Filter an access log by path to isolate sensitive endpoints
- Recognise that a 200 on a secrets path means the data was served
- Use the user-agent column to tell a script apart from a browser
- Spot systematic key harvesting from one source
- Attribute secret exfiltration to a single IP
Skills tested
Prerequisites
- Understanding of HTTP paths and status codes
- Awareness that private keys are secret material
- Familiarity with user-agent strings
How it works
Secrets get exposed when a debug or diagnostic endpoint that was meant for internal use is reachable in production. Paths like /debug/keys and /.well-known/private-keys can serve raw private key material, and anyone who finds them can simply download it. In the logs, the theft does not look violent - it is just a series of successful GET requests - so the skill is knowing which paths should never be served and noticing who is pulling them.
Filter to path:/debug and path:/.well-known and a single source, 198.51.100.31, accounts for all of it. It requests /debug/keys in both pem and jwk formats, /.well-known/private-keys, and /debug/config, over and over, each returning 200. The status code is the proof: a 200 means the server actually returned the key material, not a 403 or 404. Two more signals confirm intent - no legitimate user visits these paths, and the request ua is python-requests, a scripted client, while every benign IP uses a real browser agent to load app pages.
This is a recurring real-world failure: a forgotten debug route leaks signing keys or TLS private keys, and an attacker scripts a loop to harvest them. The log evidence is the combination of a never-public path, a repeated 200, one source, and an automated user agent.
Common mistakes
- Treating a 200 as harmless. On a secrets path, a
200is the worst case - it means the key was served. Success here is the alarm. - Only counting requests. Several IPs are busy; only one is busy on the debug and private-keys paths.
- Ignoring the user agent. The
python-requestsagent versus the browsers is a strong tell that this source is a script, not a person. - Submitting the path instead of the IP. The question asks who is harvesting the keys - the single source address.
How to defend against it
Exposed secrets are a configuration and code problem first, but detection on the access log catches active harvesting. Treat any hit on a secrets path as an incident.
- Never expose debug, diagnostic, or key-serving endpoints in production - remove them or gate them behind strong auth and an internal network.
- Alert on any request to paths that serve secrets (
/debug,/.well-known/private-keys), especially one returning200. - Rotate any key material that was reachable, and assume anything served at
200is compromised. - Block or rate-limit automated clients hammering sensitive paths and review non-browser user agents.