russian.rt.com robots.txt

Russian.rt.com has robots.txt file. Http response code is 200.
Robots size is small. It is about 99 kb and 41 rows.

russian.rt.com has sitemap file https://russian.rt.com/sitemap.xml. Main host for russian.rt.com is https://russian.rt.com.

russian.rt.com has separate rules for googlebot and yandexbot and also has rules for all bots (User-Agent=*).
Array
(
    [yandex] => Array
        (
            [disallow] => Array
                (
                    [0] => /inotv/print
                    [1] => /search
                    [2] => *?
                    [3] => /c/
                    [4] => /nbc/
                    [5] => /nbc-stats/
                    [6] => /amp*
                    [7] => /quiz/*
                )

            [allow] => Array
                (
                    [0] => *css?
                    [1] => /static/fonts/icon/
                    [2] => /static/css/
                    [3] => /static/js/
                )

        )

    [*] => Array
        (
            [host] => https://russian.rt.com
            [sitemap] => Array
                (
                    [0] => https://russian.rt.com/sitemap.xml
                )

            [disallow] => Array
                (
                    [0] => /inotv/print
                    [1] => /search
                    [2] => /c/
                    [3] => /nbc/
                    [4] => /nbc-stats/
                    [5] => *?
                    [6] => /widget/
                    [7] => /quiz/*
                )

            [allow] => Array
                (
                    [0] => *css?
                    [1] => /static/fonts/icon/
                    [2] => /static/css/
                    [3] => /static/js/
                )

        )

    [googlebot] => Array
        (
            [disallow] => Array
                (
                    [0] => /inotv/print
                    [1] => /search
                    [2] => /c/
                    [3] => /nbc/
                    [4] => /nbc-stats/
                    [5] => *?
                    [6] => /widget/
                    [7] => /quiz/*
                )

            [allow] => Array
                (
                    [0] => *css?
                    [1] => /static/fonts/icon/
                    [2] => /static/css/
                    [3] => /static/js/
                )

        )

)

User-agent: Yandex
Disallow: /inotv/print
Disallow: /search
Disallow: *?
Disallow: /c/
Disallow: /nbc/
Disallow: /nbc-stats/
Disallow: /amp*
Disallow: /quiz/*
Allow: *css?
Allow: /static/fonts/icon/
Allow: /static/css/
Allow: /static/js/
Host: https://russian.rt.com

User-agent: Googlebot
Disallow: /inotv/print
Disallow: /search
Disallow: /c/
Disallow: /nbc/
Disallow: /nbc-stats/
Disallow: *?
Disallow: /widget/
Disallow: /quiz/*
Allow: *css?
Allow: /static/fonts/icon/
Allow: /static/css/
Allow: /static/js/

Sitemap: https://russian.rt.com/sitemap.xml

User-agent: *
Disallow: /inotv/print
Disallow: /search
Disallow: /c/
Disallow: /nbc/
Disallow: /nbc-stats/
Disallow: *?
Disallow: /widget/
Disallow: /quiz/*
Allow: *css?
Allow: /static/fonts/icon/
Allow: /static/css/
Allow: /static/js/
  • код 301 http://russian.rt.com/robots.txt
  • код 200 https://russian.rt.com/robots.txt