Web & Site search engine php script, FreeThis is just a basic WEB search engine, like google but not powered by Google. You can limit it to your site in the config file and if using web search you can limit sites from your search engine {eg remove spam sites, or another site offering what you offer} Like the last script it is pretty much drop and ready. No install needed, and if you want to change stuff check config.php. It uses very little bandwidth and you are free to add Google adsense.
_http://hotscripts.com/jump.php?listing_id=63519&jump_type=0
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Robots.txt parserGive this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).
_http://hotscripts.com/jump.php?listing_id=53640&jump_type=0