Toggle navigation
Robots.txt
Php library
Github
Search
Stars
Forks
Issues
klipfish.de
robots.txt
Klipfish.de
has robots.txt file. Http response code is 200.
Robots size is very small. It is about 5 kb and 2 rows.
Host
directive is not presented for domain klipfish.de in robots.txt file.
Result of parse
Raw robots.txt content
Response analysis
Array ( [mj12bot] => Array ( [crawl-delay] => 19 ) )
User-Agent: MJ12bot Crawl-Delay: 19
код 200
http://klipfish.de/robots.txt