First though, you should study how
crawlers work before making it in PHP. If you are trying to crawl other websites, you should first learn cURL which is a good php extension that you can use to browse other sites through php. Reading the links might be trick thoguh because you have to see if there was a nofollow or not but just take it on step at a time. If you are trying to crawl your...