meta.ua robots.txt

Meta.ua has robots.txt file. Http response status is 200.
Robots size is very small. It is about 36 kb and 13 rows.

Robots.txt for User-Agent=* contains only disallow directives.

Main host for meta.ua is https://meta.ua.

Array
(
    [*] => Array
        (
            [disallow] => Array
                (
                    [0] => *go.asp*
                    [1] => *search_sett.asp*
                    [2] => *adv_search.asp*
                    [3] => *search.asp*
                    [4] => /cdn-cgi/
                    [5] => */monthNewsDays/*
                    [6] => /secured/
                    [7] => /uk/secured/
                )

            [host] => https://meta.ua
        )

    [yandex] => Array
        (
            [disallow] => Array
                (
                    [0] => /finance/
                    [1] => /uk/finance/
                )

        )

)

User-agent: *
Disallow: *go.asp*
Disallow: *search_sett.asp*
Disallow: *adv_search.asp*
Disallow: *search.asp*
Disallow: /cdn-cgi/
Disallow: */monthNewsDays/*
Disallow: /secured/
Disallow: /uk/secured/

User-agent: Yandex
Disallow:/finance/
Disallow: /uk/finance/

Host: https://meta.ua
  • код 301 http://meta.ua/robots.txt
  • код 200 https://meta.ua/robots.txt