Robots.txt is a file which instructs Search engine crawlers, spiders and robots which part of the site to be indexed and which part of the site not be indexed. Basically...

The post How to stop Search Engines from crawling a file or folder using Robots.txt appeared first on Techgist - Restore Firefox Bookmarks, Apple ipad Tablet, Wordpress Database Backup, Wordpress Themes, Wordpress Manual Backup.