In order to avoid unwanted articles from the search indexes, webmasters can instruct spiders never to crawl selected files or directories with the regular robots.txt file in the root directory in the domain. On top of that, a web site may be explicitly excluded from a internet search engine's databases https://www.facebook.com/acourseinmiraclesdavid/