As smart as Google’s search engine spiders are, even they can miss pages on your site while indexing for search results. Maybe you have moved a link to content so that it’s not easily accessible. Or, it could be possible your site is too big for Google to crawl without pulling all your server’s resources - not pretty!
The solution is simple: a sitemap. There are several types of sitemaps you can use on your Drupal site, but we will cover the most important: the XML sitemap. XML sitemaps are designed to be used by search engines for indexing your pages. Here is a refined definition according to www.sitemaps.org:
Sitemaps are an easy way for Webmasters to inform search engines about pages on their sites that are available for crawling. In its simplest form, a Sitemap is an XML file that lists URLs for a site along with additional metadata about each URL (when it was last updated, how often it usually changes, and how important it is, relative to other URLs in the site) so that search engines can more intelligently crawl the site. Web crawlers usually discover pages from links within the site and from other sites. Sitemaps supplement this data to allow crawlers that support Sitemaps to pick up all URLs in the Sitemap and learn about those URLs using the associated metadata.
Using a sitemap does not guarantee that every page on your site will be included in the search engines.
Rather, it helps the search engines find more of your pages.
Submitting an XML sitemap to Google will significantly increase the number of pages when you do a site:search. The keyword site: searches show you how many pages of your site are included in the search engine index, as shown in the following screenshot:
Go read our most recent installation of the Drupal SEO Guide for detailed instructions on how to install, enable, and configure your XML sitemap.
Too busy to optimize your own website? Let the experts at Volacci help you out.