lsu.edu robots.txt
Lsu.edu has robots.txt file. Http response status is 200.
Robots size is very small. It is about 25 kb and 11 rows.
Robots.txt for User-Agent=* contains only disallow directives.
Host directive is not presented for domain lsu.edu in robots.txt file.
Robots size is very small. It is about 25 kb and 11 rows.
Robots.txt for User-Agent=* contains only disallow directives.
Host directive is not presented for domain lsu.edu in robots.txt file.
Array
(
[semrushbot] => Array
(
[disallow] => Array
(
[0] => /
)
)
[*] => Array
(
[disallow] => Array
(
[0] => /bookmgr-cgi/
[1] => /webtest/
[2] => /cgi-bin/
[3] => /hodold/
[4] => /reports/
[5] => /hod/
[6] => /HOD/
[7] => /_*
)
)
)
User-agent: SemrushBot Disallow: / User-agent: * Disallow: /bookmgr-cgi/ Disallow: /webtest/ Disallow: /cgi-bin/ Disallow: /hodold/ Disallow: /reports/ Disallow: /hod/ Disallow: /HOD/ Disallow: /_*
- код 301 http://lsu.edu/robots.txt
- код 200 https://lsu.edu/robots.txt