RobotsDisallowed
danielmiessler/RobotsDisallowed
A curated list of the most common and most interesting robots.txt disallowed directories.
1,486stars
Forks
304
Open issues
5
Watchers
1,486
Size
211.0 MB
Shell
Created: Sep 21, 2015
Updated: Feb 22, 2026
Last push: Aug 22, 2022