Seolik.ru has robots.txt file. Http response status is 200.
Robots size is small. It is about 102 kb and 30 rows.

Robots.txt for User-Agent=* contains only disallow directives.

Main host for seolik.ru is https://seolik.ru.

Array
(
    [*] => Array
        (
            [disallow] => Array
                (
                    [0] => /informer
                    [1] => /asite
                    [2] => /infodata
                    [3] => /infodata2
                )

            [crawl-delay] => 30
            [sitemap] => Array
                (
                    [0] => https://seolik.ru/sitemap.xml
                    [1] => https://seolik.ru/sitemaps/users.xml
                )

            [host] => https://seolik.ru
        )

    [yandex] => Array
        (
            [crawl-delay] => 30
            [disallow] => Array
                (
                    [0] => /informer
                    [1] => /asite
                    [2] => /infodata
                    [3] => /infodata2
                    [4] => /mytools/search-traffic
                    [5] => /mytools/search-traffic?site=
                )

            [clean-param] => Array
                (
                    [0] => ifrom /login
                    [1] => price /cost-site
                    [2] => url /age
                    [3] => url /whois
                    [4] => url /security-headers
                    [5] => url /check-blocked-sites
                    [6] => url /ssl-check
                    [7] => url /speed
                    [8] => word /top-yandex
                    [9] => link /index-page-yandex
                    [10] => url /links
                )

        )

)
User-agent: *
Disallow: /informer
Disallow: /asite
Disallow: /infodata
Disallow: /infodata2
Crawl-delay: 30
Sitemap: https://seolik.ru/sitemap.xml
Sitemap: https://seolik.ru/sitemaps/users.xml

User-agent: Yandex
Crawl-delay: 30
Host: https://seolik.ru
Disallow: /informer
Disallow: /asite
Disallow: /infodata
Disallow: /infodata2
Disallow: /mytools/search-traffic
Disallow: /mytools/search-traffic?site=
Clean-param: ifrom /login
Clean-param: price /cost-site
Clean-param: url /age
Clean-param: url /whois
Clean-param: url /security-headers
Clean-param: url /check-blocked-sites
Clean-param: url /ssl-check
Clean-param: url /speed
Clean-param: word /top-yandex
Clean-param: link /index-page-yandex
Clean-param: url /links

Sitemap: https://seolik.ru/sitemap.xml
Sitemap: https://seolik.ru/sitemaps/users.xml




  • код 200 http://seolik.ru/robots.txt