Example of library using:

$robotsTxtContent = file_get_contents('http://example.com/robots.txt');
$parser = new RobotsTxtParser($robotsTxtContent);
print_r($parser->getRules());

 

 

The latest release notes:

 

Fork me on GitHub