New-rutor.org has robots.txt file. Http response code is 200.
Robots size is small. It is about 122 kb and 23 rows.

new-rutor.org has great amount of sitemap files — 12: http://new-rutor.org/sitemaps/new-rutor.org/new.xml, http://new-rutor.org/sitemaps/new-rutor.org/1.xml, http://new-rutor.org/sitemaps/new-rutor.org/2.xml and others. Robots.txt for User-Agent=* contains only disallow directives.

Array
(
    [*] => Array
        (
            [crawl-delay] => 0.3
            [disallow] => Array
                (
                    [0] => /new11/
                    [1] => /newCache/
                    [2] => /topCache/
                    [3] => /inc/checkcache/
                    [4] => /inc/recache/
                    [5] => /redirect/
                    [6] => /parse/d.rutor.org/download/
                )

            [clean-param] => Array
                (
                    [0] => osCsid&recache&checkcache
                )

            [host] => new-rutor.org
            [sitemap] => Array
                (
                    [0] => http://new-rutor.org/sitemaps/new-rutor.org/new.xml
                    [1] => http://new-rutor.org/sitemaps/new-rutor.org/1.xml
                    [2] => http://new-rutor.org/sitemaps/new-rutor.org/2.xml
                    [3] => http://new-rutor.org/sitemaps/new-rutor.org/3.xml
                    [4] => http://new-rutor.org/sitemaps/new-rutor.org/4.xml
                    [5] => http://new-rutor.org/sitemaps/new-rutor.org/5.xml
                    [6] => http://new-rutor.org/sitemaps/new-rutor.org/6.xml
                    [7] => http://new-rutor.org/sitemaps/new-rutor.org/7.xml
                    [8] => http://new-rutor.org/sitemaps/new-rutor.org/8.xml
                    [9] => http://new-rutor.org/sitemaps/new-rutor.org/9.xml
                    [10] => http://new-rutor.org/sitemaps/new-rutor.org/10.xml
                    [11] => http://new-rutor.org/sitemaps/new-rutor.org/11.xml
                )

        )

)
User-agent: *
Crawl-delay: 0.3
Disallow: /new11/
Disallow: /newCache/
Disallow: /topCache/
Disallow: /inc/checkcache/
Disallow: /inc/recache/
Disallow: /redirect/
Disallow: /parse/d.rutor.org/download/
Clean-param: osCsid&recache&checkcache
Host: new-rutor.org

Sitemap: http://new-rutor.org/sitemaps/new-rutor.org/new.xml
Sitemap: http://new-rutor.org/sitemaps/new-rutor.org/1.xml
Sitemap: http://new-rutor.org/sitemaps/new-rutor.org/2.xml
Sitemap: http://new-rutor.org/sitemaps/new-rutor.org/3.xml
Sitemap: http://new-rutor.org/sitemaps/new-rutor.org/4.xml
Sitemap: http://new-rutor.org/sitemaps/new-rutor.org/5.xml
Sitemap: http://new-rutor.org/sitemaps/new-rutor.org/6.xml
Sitemap: http://new-rutor.org/sitemaps/new-rutor.org/7.xml
Sitemap: http://new-rutor.org/sitemaps/new-rutor.org/8.xml
Sitemap: http://new-rutor.org/sitemaps/new-rutor.org/9.xml
Sitemap: http://new-rutor.org/sitemaps/new-rutor.org/10.xml
Sitemap: http://new-rutor.org/sitemaps/new-rutor.org/11.xml
  • код 200 http://new-rutor.org/robots.txt