RobotsDisallowed
danielmiessler/RobotsDisallowed
A curated list of the most common and most interesting robots.txt disallowed directories.
1,484stars
Forks
304
Open issues
5
Watchers
1,484
Size
211.0 MB
Shell
Created: Sep 21, 2015
Updated: Apr 10, 2026
Last push: Aug 22, 2022