Rozetka.ua has robots.txt file. Http response code is 200.
Robots size is small. It is about 82 kb and 29 rows.

Host directive is not presented for domain rozetka.ua in robots.txt file.
Array
(
    [*] => Array
        (
            [disallow] => Array
                (
                    [0] => /cgi-bin
                    [1] => /search/
                    [2] => /*sort=*
                    [3] => /*utm_medium=rooms*
                    [4] => /*promotions/top_section*
                    [5] => /*tab=*
                    [6] => /*gclid=*
                    [7] => /*yclid=*
                    [8] => /*/delivery/
                    [9] => /*utm_trafmaguid=*
                    [10] => /comparison/
                    [11] => /*/comparison/
                    [12] => /landing_pages/*
                    [13] => /*kits_costs*
                    [14] => /landing-pages/*
                    [15] => /lazy_load/*
                    [16] => /*/recent.rss
                    [17] => /ua/
                    [18] => /*?*
                )

            [allow] => Array
                (
                    [0] => /recent_recommends/action=getGoodsDetailsJSON/?goods_ids
                )

        )

    [yahoo] => Array
        (
            [disallow] => Array
                (
                    [0] => /
                )

        )

    [mj12bot] => Array
        (
            [disallow] => Array
                (
                    [0] => /
                )

        )

    [mediapartners-google] => Array
        (
            [disallow] => Array
                (
                    [0] => /
                )

        )

    [ahrefsbot] => Array
        (
            [disallow] => Array
                (
                    [0] => /
                )

        )

)
User-Agent: *

Disallow: /cgi-bin
Disallow: /search/
Disallow: /*sort=*
Disallow: /*utm_medium=rooms*
Disallow: /*promotions/top_section*
Disallow: /*tab=*
Disallow: /*gclid=*
Disallow: /*yclid=*
Disallow: /*/delivery/
Disallow: /*utm_trafmaguid=*
Disallow: /comparison/
Disallow: /*/comparison/
Disallow: /landing_pages/*
Disallow: /*kits_costs*
Disallow: /landing-pages/*
Disallow: /lazy_load/*
Disallow: /*/recent.rss
Disallow: /ua/
Allow: /recent_recommends/action=getGoodsDetailsJSON/?goods_ids
Disallow: /*?*

User-agent: Yahoo
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: Mediapartners-Google
Disallow: /

User-agent: AhrefsBot
Disallow: /
  • код 301 http://rozetka.ua/robots.txt
  • код 200 https://rozetka.com.ua/robots.txt