Cryptorevolt.site has robots.txt file. Http response code is 200.
Robots size is very small. It is about 34 kb and 15 rows.

Robots.txt for User-Agent=* contains only disallow directives.

Host directive is not presented for domain cryptorevolt.site in robots.txt file.
Array
(
    [ia_archiver] => Array
        (
        )

    [*] => Array
        (
            [disallow] => Array
                (
                    [0] => /note/
                    [1] => /search.php
                    [2] => /click.php
                    [3] => /search_caf.php
                    [4] => /t.php
                    [5] => /exitpage/
                    [6] => /popup/
                    [7] => /r.php
                    [8] => /secondary_feed/
                )

        )

    [mj12bot] => Array
        (
            [disallow] => Array
                (
                    [0] => /
                )

        )

)

User-Agent: ia_archiver
Disallow:
 
User-Agent: *
Disallow: /note/
Disallow: /search.php
Disallow: /click.php
Disallow: /search_caf.php
Disallow: /t.php
Disallow: /exitpage/
Disallow: /popup/
Disallow: /r.php
Disallow: /secondary_feed/

User-agent: MJ12bot
Disallow: /



  • код 200 http://cryptorevolt.site/robots.txt