Ntv.ru has robots.txt file. Http response status is 200.
Robots size is small. It is about 63 kb and 29 rows.

ntv.ru has sitemap file https://www.ntv.ru/exp/video_sitemap.xml. Robots.txt for User-Agent=* contains only disallow directives.

Main host for ntv.ru is https://www.ntv.ru/.

Array
(
    [telegrambot] => Array
        (
            [disallow] => Array
                (
                    [0] => /video
                    [1] => /embed
                    [2] => /login
                    [3] => /search
                    [4] => /forms
                    [5] => /ntv
                    [6] => /finder
                    [7] => /private
                )

        )

    [*] => Array
        (
            [disallow] => Array
                (
                    [0] => /login
                    [1] => /search
                    [2] => /forms
                    [3] => /ntv
                    [4] => /finder
                    [5] => /private
                    [6] => /rest
                    [7] => /foto
                )

            [sitemap] => Array
                (
                    [0] => https://www.ntv.ru/exp/video_sitemap.xml
                )

            [host] => https://www.ntv.ru/
        )

    [trendkite-akashic-crawler] => Array
        (
            [disallow] => Array
                (
                    [0] => /
                )

        )

    [petalbot] => Array
        (
            [disallow] => Array
                (
                    [0] => /
                )

        )

)
User-agent: TelegramBot
Disallow: /video
Disallow: /embed
Disallow: /login
Disallow: /search
Disallow: /forms
Disallow: /ntv
Disallow: /finder
Disallow: /private

User-agent: *
Disallow: /login
Disallow: /search
Disallow: /forms
Disallow: /ntv
Disallow: /finder
Disallow: /private
Disallow: /rest
Disallow: /foto

User-agent: trendkite-akashic-crawler
Disallow: /

User-agent: PetalBot
Disallow: /

Sitemap: https://www.ntv.ru/exp/video_sitemap.xml

Host: https://www.ntv.ru/
  • код 301 http://ntv.ru/robots.txt
  • код 200 https://www.ntv.ru/robots.txt