mobile.free.fr robots.txt
Mobile.free.fr has robots.txt file. Http response code is 200.
Robots size is small. It is about 164 kb and 54 rows.
mobile.free.fr has sitemap file https://mobile.free.fr/sitemap.xml. Host directive is not presented for domain mobile.free.fr in robots.txt file.
Robots size is small. It is about 164 kb and 54 rows.
mobile.free.fr has sitemap file https://mobile.free.fr/sitemap.xml. Host directive is not presented for domain mobile.free.fr in robots.txt file.
Array
(
[googlebot] => Array
(
)
[googlebot-image] => Array
(
)
[mediapartners-google] => Array
(
)
[googlebot-news] => Array
(
)
[googlebot-video] => Array
(
)
[adsbot-google] => Array
(
)
[adsbot-google-mobile] => Array
(
)
[adsbot-google-mobile-apps] => Array
(
)
[storebot-google] => Array
(
)
[bingbot] => Array
(
)
[adidxbot] => Array
(
)
[duckduckbot] => Array
(
)
[slurp] => Array
(
)
[qwantify] => Array
(
)
[pinterest] => Array
(
[allow] => Array
(
[0] => /
)
)
[*] => Array
(
[disallow] => Array
(
[0] => /docs/veepee/Recapitulatif_contractuel_serie_speciale.pdf
[1] => /docs/veepee/FIS_veepee.pdf
[2] => /informations/avis-free-mobile?page=
[3] => /shop/products/?service=getCouponODR*
[4] => /*contentOnly*
[5] => /shop?from=selfcare&userShops=
[6] => /shop?from=subscribe&_=
)
[allow] => Array
(
[0] => /informations/avis-free-mobile?page=1
[1] => /shop?from=subscribe&orderBy=price_desc
[2] => /shop
)
[sitemap] => Array
(
[0] => https://mobile.free.fr/sitemap.xml
)
)
[buenibot] => Array
(
[disallow] => Array
(
[0] => /
)
)
[etaospider] => Array
(
[disallow] => Array
(
[0] => /
)
)
[yandex] => Array
(
[disallow] => Array
(
[0] => /
)
)
[baiduspider] => Array
(
[disallow] => Array
(
[0] => /
)
)
[baiduspider-image] => Array
(
[disallow] => Array
(
[0] => /
)
)
[360spider] => Array
(
[disallow] => Array
(
[0] => /
)
)
[mj12bot] => Array
(
[disallow] => Array
(
[0] => /
)
)
[cazoodlebot] => Array
(
[disallow] => Array
(
[0] => /
)
)
[haosouspider] => Array
(
[disallow] => Array
(
[0] => /
)
)
[ia_archiver] => Array
(
[disallow] => Array
(
[0] => /
)
)
[wget] => Array
(
[disallow] => Array
(
[0] => /
)
)
[turnitinbot] => Array
(
[disallow] => Array
(
[0] => /
)
)
[httrack] => Array
(
[disallow] => Array
(
[0] => /
)
)
)
User-Agent: Googlebot User-Agent: Googlebot-Image User-Agent: Mediapartners-Google User-Agent: Googlebot-News User-Agent: Googlebot-Video User-Agent: AdsBot-Google User-Agent: AdsBot-Google-Mobile User-Agent: AdsBot-Google-Mobile-Apps User-Agent: Storebot-Google User-Agent: Bingbot User-Agent: AdIdxBot User-Agent: DuckDuckBot User-Agent: Slurp User-Agent: Qwantify User-Agent: Pinterest Allow: / User-Agent: * Disallow: /docs/veepee/Recapitulatif_contractuel_serie_speciale.pdf Disallow: /docs/veepee/FIS_veepee.pdf Disallow: /informations/avis-free-mobile?page= Allow: /informations/avis-free-mobile?page=1 Disallow: /shop/products/?service=getCouponODR* Disallow: /*contentOnly* Disallow: /shop?from=selfcare&userShops= Disallow: /shop?from=subscribe&_= Allow: /shop?from=subscribe&orderBy=price_desc Allow: /shop User-Agent: buenibot Disallow: / User-Agent: EtaoSpider Disallow: / User-Agent: Yandex Disallow: / User-Agent: Baiduspider Disallow: / User-Agent: Baiduspider-image Disallow: / User-Agent: 360Spider Disallow: / User-Agent: MJ12bot Disallow: / User-Agent: CazoodleBot Disallow: / User-Agent: HaosouSpider Disallow: / User-Agent: ia_archiver Disallow: / User-Agent: Wget Disallow: / User-Agent: TurnitinBot Disallow: / User-Agent: HTTrack Disallow: / Sitemap: https://mobile.free.fr/sitemap.xml
- код 200 http://mobile.free.fr/robots.txt