XML Sitemaps; the answer to your indexing problems?
XML sitemaps are widely used, they are supported by all the major search engines and they pretty much guarantee that your pages will get crawled. Is this an SEO tool which is too good to be true?
Back in June 2005 Google introduced a new feature called Google sitemaps, describing it as a beta "ecosystem" that may help webmasters with two current challenges: keeping Google informed about all of your new web pages or updates, and increasing the coverage of your web pages in the Google index.
Since Google released sitemaps under a creative commons licence, all the major search engines were soon announcing support for the sitemap feeds which the Google Sitemap Generator produced.
‘This is great!’ I hear you cry, ‘Every one of my pages is going to be indexed!’ and you’d be right. The engines are making no guarantees, but they will index pretty much every page that they know about, assuming that the page has not been excluded from the index by the webmaster. And that’s where the problem lies.
The major search engine algorithms are all heavily weighted towards link analysis. Using an XML sitemap may cause two problems with regards to this:
- If a search engine has a list of every page on the site in the form of an XML feed, it becomes almost impossible to tell whether the pages have been found organically or solely through the XML feed. This sort of information is a huge boon when discovering which sections of a site is doing well and which parts badly, and why.
- Finding an entire site through an XML sitemap means that any page found which had not previously been discovered organically by the search engine will be seen as a stand alone page with no inbound links. These ‘orphaned’ pages are very unlikely to rank highly enough to convert and this could lead to a large number of pages being placed in supplemental indices.
XML sitemaps are not an SEO tool. If your page isn’t getting indexed organically then it isn’t going to rank well either. Unless you have a niche keyword for which only half a dozen pages are being found, there is no advantage to being indexed if your site is never going to be returned for a search.
My personal experience has shown that it may take longer for sites to move out of supplemental indices than it would have taken to get them into the main index initially but, with supplemental indices being less transparent than they were a year ago, it is hard to verify whether this observed correlation is indicative of a genuine issue or merely co-incidental.
All an XML feed is doing for you is removing your opportunity to assess which pages have organic indexing issues. A good internal linking structure, including an HTML sitemap and developing sensible link building strategies will provide a much better return than an XML sitemap ever will.
Tags: creative commons, sitemaps, supplemental