Haz.de has robots.txt file. Http response code is 200.
Robots size is very small. It is about 34 kb and 10 rows.

Robots.txt for User-Agent=* contains only disallow directives.

Host directive is not presented for domain haz.de in robots.txt file.
Array
(
    [*] => Array
        (
            [disallow] => Array
                (
                    [0] => /disabledFunctionsForCrawlers.chunk.js
                    [1] => /mandanten/
                    [2] => /mediabox/
                    [3] => /politik/politik-extern/
                    [4] => /wirtschaft/wirtschaft-extern
                    [5] => /suche/
                    [6] => /ellipsis-preview/
                    [7] => /pf/api/v3/
                    [8] => /bundles/
                )

        )

)
User-agent: *
Disallow: /disabledFunctionsForCrawlers.chunk.js
Disallow: /mandanten/
Disallow: /mediabox/
Disallow: /politik/politik-extern/
Disallow: /wirtschaft/wirtschaft-extern
Disallow: /suche/
Disallow: /ellipsis-preview/
Disallow: /pf/api/v3/

Disallow: /bundles/
  • код 301 http://haz.de/robots.txt
  • код 301 http://www.haz.de/robots.txt
  • код 200 https://www.haz.de/robots.txt