digitalspy.com robots.txt
Digitalspy.com has robots.txt file. Http response code is 200.
Robots size is small. It is about 263 kb and 81 rows.
Host directive is not presented for domain digitalspy.com in robots.txt file.
Robots size is small. It is about 263 kb and 81 rows.
Host directive is not presented for domain digitalspy.com in robots.txt file.
Array
(
[ccbot] => Array
(
[disallow] => Array
(
[0] => /
)
)
[chatgpt-user] => Array
(
)
[gptbot] => Array
(
)
[oai-searchbot] => Array
(
[disallow] => Array
(
[0] => /
)
)
[*] => Array
(
[disallow] => Array
(
[0] => /ajax/
[1] => /auth/
[2] => /au/ajax/
[3] => /cn/ajax/
[4] => /dk/ajax/
[5] => /en/ajax/
[6] => /es/ajax/
[7] => /in/ajax/
[8] => /it/ajax/
[9] => /jp/ajax/
[10] => /ng/ajax/
[11] => /nl/ajax/
[12] => /no/ajax/
[13] => /se/ajax/
[14] => /tw/ajax/
[15] => /ua/ajax/
[16] => /uk/ajax/
[17] => /landing-feed/
[18] => /oauth/
[19] => /preview/
[20] => /au/preview/
[21] => /cn/preview/
[22] => /dk/preview/
[23] => /en/preview/
[24] => /es/preview/
[25] => /in/preview/
[26] => /it/preview/
[27] => /jp/preview/
[28] => /ng/preview/
[29] => /nl/preview/
[30] => /no/preview/
[31] => /se/preview/
[32] => /tw/preview/
[33] => /ua/preview/
[34] => /uk/preview/
[35] => /search
[36] => /transporter/
[37] => /images/
[38] => /*moapt-data.js
[39] => /_p/afflink/
)
[allow] => Array
(
[0] => /ajax/content-product/
[1] => /au/ajax/content-product/
[2] => /cn/ajax/content-product/
[3] => /dk/ajax/content-product/
[4] => /en/ajax/content-product/
[5] => /es/ajax/content-product/
[6] => /in/ajax/content-product/
[7] => /it/ajax/content-product/
[8] => /jp/ajax/content-product/
[9] => /ng/ajax/content-product/
[10] => /nl/ajax/content-product/
[11] => /no/ajax/content-product/
[12] => /se/ajax/content-product/
[13] => /tw/ajax/content-product/
[14] => /ua/ajax/content-product/
[15] => /uk/ajax/content-product/
[16] => /ajax/contentmedia/
[17] => /au/ajax/contentmedia/
[18] => /cn/ajax/contentmedia/
[19] => /dk/ajax/contentmedia/
[20] => /en/ajax/contentmedia/
[21] => /es/ajax/contentmedia/
[22] => /in/ajax/contentmedia/
[23] => /it/ajax/contentmedia/
[24] => /jp/ajax/contentmedia/
[25] => /ng/ajax/contentmedia/
[26] => /nl/ajax/contentmedia/
[27] => /no/ajax/contentmedia/
[28] => /se/ajax/contentmedia/
[29] => /tw/ajax/contentmedia/
[30] => /ua/ajax/contentmedia/
[31] => /uk/ajax/contentmedia/
)
[sitemap] => Array
(
[0] => https://www.digitalspy.com/sitemap_index.xml
[1] => https://www.digitalspy.com/sitemap_google_news.xml
)
)
)
User-agent: CCBot Disallow: / User-agent: ChatGPT-User User-agent: GPTBot User-agent: OAI-SearchBot Disallow: / User-agent: * Disallow: /ajax/ Disallow: /auth/ Disallow: /au/ajax/ Disallow: /cn/ajax/ Disallow: /dk/ajax/ Disallow: /en/ajax/ Disallow: /es/ajax/ Disallow: /in/ajax/ Disallow: /it/ajax/ Disallow: /jp/ajax/ Disallow: /ng/ajax/ Disallow: /nl/ajax/ Disallow: /no/ajax/ Disallow: /se/ajax/ Disallow: /tw/ajax/ Disallow: /ua/ajax/ Disallow: /uk/ajax/ Disallow: /landing-feed/ Disallow: /oauth/ Disallow: /preview/ Disallow: /au/preview/ Disallow: /cn/preview/ Disallow: /dk/preview/ Disallow: /en/preview/ Disallow: /es/preview/ Disallow: /in/preview/ Disallow: /it/preview/ Disallow: /jp/preview/ Disallow: /ng/preview/ Disallow: /nl/preview/ Disallow: /no/preview/ Disallow: /se/preview/ Disallow: /tw/preview/ Disallow: /ua/preview/ Disallow: /uk/preview/ Disallow: /search Disallow: /transporter/ Disallow: /images/ Disallow: /*moapt-data.js Disallow: /_p/afflink/ Allow: /ajax/content-product/ Allow: /au/ajax/content-product/ Allow: /cn/ajax/content-product/ Allow: /dk/ajax/content-product/ Allow: /en/ajax/content-product/ Allow: /es/ajax/content-product/ Allow: /in/ajax/content-product/ Allow: /it/ajax/content-product/ Allow: /jp/ajax/content-product/ Allow: /ng/ajax/content-product/ Allow: /nl/ajax/content-product/ Allow: /no/ajax/content-product/ Allow: /se/ajax/content-product/ Allow: /tw/ajax/content-product/ Allow: /ua/ajax/content-product/ Allow: /uk/ajax/content-product/ Allow: /ajax/contentmedia/ Allow: /au/ajax/contentmedia/ Allow: /cn/ajax/contentmedia/ Allow: /dk/ajax/contentmedia/ Allow: /en/ajax/contentmedia/ Allow: /es/ajax/contentmedia/ Allow: /in/ajax/contentmedia/ Allow: /it/ajax/contentmedia/ Allow: /jp/ajax/contentmedia/ Allow: /ng/ajax/contentmedia/ Allow: /nl/ajax/contentmedia/ Allow: /no/ajax/contentmedia/ Allow: /se/ajax/contentmedia/ Allow: /tw/ajax/contentmedia/ Allow: /ua/ajax/contentmedia/ Allow: /uk/ajax/contentmedia/ Sitemap: https://www.digitalspy.com/sitemap_index.xml Sitemap: https://www.digitalspy.com/sitemap_google_news.xml
- код 200 http://digitalspy.com/robots.txt
- код 200 http://digitalspy.com/robots.txt