has robots.txt file. Http response code is 200.
Robots size is very small. It is about 3 kb and 2 rows.
The Domain is inaccessible for indexation by search robots because robots.txt has directive "
directive is not presented for domain newsru.co.il in robots.txt file.
Result of parse
Raw robots.txt content
Array ( [*] => Array ( [disallow] => Array (  => / ) ) )
User-agent: * Disallow: /