martech.zone robots.txt

Martech.zone has robots.txt file. Http response code is 200.
Robots size is small. It is about 116 kb and 31 rows.

martech.zone has sitemap file https://martech.zone/sitemap_index.xml. Host directive is not presented for domain martech.zone in robots.txt file. 
Array
(
    [*] => Array
        (
            [crawl-delay] => 10
            [disallow] => Array
                (
                    [0] => /?s=*
                    [1] => /?search=*
                    [2] => /*?mailpoet_router*
                    [3] => /*&mailpoet_router*
                    [4] => /refer/*
                    [5] => /wp-admin/
                    [6] => /wp-login.php*
                    [7] => /xmlrpc.php*
                    [8] => /readme.html*
                    [9] => /wp-content/plugins/
                    [10] => /wp-content/themes/
                    [11] => /trackback/
                    [12] => /comments/feed/
                    [13] => */zlib/
                    [14] => /*/feed/$
                    [15] => /*?pdfemb-data=*
                    [16] => /*?ignorenitro=*
                    [17] => /*?replytocom=
                )

            [allow] => Array
                (
                    [0] => /wp-admin/admin-ajax.php
                    [1] => /wp-content/uploads/
                )

            [sitemap] => Array
                (
                    [0] => https://martech.zone/sitemap_index.xml
                )

        )

)

User-agent: *
# Slower crawl rate for Bing, Yandex, and generic bots
Crawl-delay: 10

# Block Search parameters but keep /search/ paths open
Disallow: /?s=*
Disallow: /?search=*

# Block Mailpoet archives
Disallow: /*?mailpoet_router*
Disallow: /*&mailpoet_router*

# Block Referral and Affiliate redirects
Disallow: /refer/*

# Block heavy WordPress system files/folders
Disallow: /wp-admin/
Disallow: /wp-login.php*
Disallow: /xmlrpc.php*
Disallow: /readme.html*
Disallow: /wp-content/plugins/
Disallow: /wp-content/themes/
Disallow: /trackback/
Disallow: /comments/feed/

# Block specific heavy patterns you identified
Disallow: */zlib/
Disallow: /*/feed/$
Disallow: /*?pdfemb-data=*
Disallow: /*?ignorenitro=*

# Block common crawler traps (Comment replies)
Disallow: /*?replytocom=

# Allow specific necessary files
Allow: /wp-admin/admin-ajax.php
Allow: /wp-content/uploads/

Sitemap: https://martech.zone/sitemap_index.xml
  • код 301 http://martech.zone/robots.txt
  • код 200 https://martech.zone/robots.txt