Week 9: 10 Things to do Before the Spiders Come
Tip #9: The robots.txt
This file allows you to filter out the bots that crawl your website, plus you can restrict certain URLs that do not need to be indexed. This is a tremendous resource if used properly.
by Ben Finklea on July 9, 2007 - 4:16am
Bookmark/Search this post with:
Before partnering with Volacci, our website was nowhere to be found, but after a few months with Volacci, we were all over the front pages of the major search engines.
— Justin McMinn
Full Partner
McMinn Law
About the author
Ben Finklea
Ben and the Volacci® team provide Search Engine Optimization, Paid Search, and Conversions Consulting to a varied client base - ranging from local real estate agents to Fortune 500 companies. Ben's book Drupal 6 Search Engine Optimization was released in September, 2009 and is available from Amazon.com.
Comments
Post new comment