app.zip robots.txt
App.zip has robots.txt file. Http response code is 200.
Robots size is very small. It is about 39 kb and 19 rows.
The Domain is inaccessible for indexation by search robots because robots.txt has directive "Disallow: /".
Host directive is not presented for domain app.zip in robots.txt file.
Robots size is very small. It is about 39 kb and 19 rows.
The Domain is inaccessible for indexation by search robots because robots.txt has directive "Disallow: /".
Host directive is not presented for domain app.zip in robots.txt file.
Array
(
[googlebot] => Array
(
[crawl-delay] => 10
)
[mediapartners-google] => Array
(
)
[slurp] => Array
(
[crawl-delay] => 10
)
[msnbot] => Array
(
[crawl-delay] => 10
)
[bingbot] => Array
(
[crawl-delay] => 10
)
[ia_archiver] => Array
(
[crawl-delay] => 10
)
[*] => Array
(
[disallow] => Array
(
[0] => /
)
)
)
User-agent: Googlebot Crawl-delay: 10 Disallow: User-agent: Mediapartners-Google Disallow: User-agent: Slurp Crawl-delay: 10 Disallow: User-agent: MSNBot Crawl-delay: 10 Disallow: User-agent: bingbot Crawl-delay: 10 Disallow: User-agent: ia_archiver Crawl-delay: 10 Disallow: User-agent: * Disallow: /
- код 200 http://app.zip/robots.txt