Upon the recent launch of our new LBi.com site we were alarmed to notice that Google was sending visitors to the wrong site!
As you can see below, at the time of writing, a search for [lbi.com] in google.co.uk will display a result for the Leo Baeck Institute in New York, a site about the history and culture of German speaking Jewry hosted on the domain ‘lbi.org’. The ‘sitelinks’ underneath the top result also erroneously refer and link to pages on the lbi.org domain:
This is badly wrong. As it happens, this is not a major disaster for LBi, but it could be much different for our natural search clients, who could lose significant revenues as a result of this kind of error.
So why did this happen?
There are no configurations or logical connections between the “lbi.com” site and the “lbi.org” site which could have mislead Google, leaving only two options; an error in Google code, or an error in a manually edited result – the latter of which we believe to be the most likely reason.
This is a very rare occurance that gives us an insight into the world of Google, in particular how some results are so well positioned, despite there being no ‘apparent’ reason for them to be performing so well.
We do see this from time to time, although it should be stressed that the overwhelming majority of sites will never see this kind of manual intervention, and usual best practices still apply.
One reason this result may have been singled out is due to Google’s recent focus on branded search. We suspect that brand results are one of the items currently being identified and prioritised by Google for search quality purposes.
Why would Google be manually editing search results in 2010?
Manually editing SERPS is more common than you might think. It happens for numerous reasons, from legal requests for removal of content, to handing out “black hat” SEO penalties, to delivering expected results for high volume navigational queries where, for example, a user is searching for a branded website.
Search engines have a conundrum, in that they need websites to be included in their index to attract searchers. If they remove websites for infringing terms and conditions no matter who they are, search engine users would soon get fed up and find another search engine. Likewise, if a search engine doesn’t surface expected results for a query because the site a user seeks is not optimised well enough to naturally be top of the search engine results, search engines reserve the right to manually edit results.
This introduces the potential for human error, which we believe is the case for the erroneous result demonstrated here.
Digging a little deeper:
The cached copy of this page, shown below as indexed on the 7th of August, clearly shows “lbi.com” in the cache URL, but “lbi.org” in the cache description. This is only the case for the homepage, for the phrase [lbi.com]:
The same error is evidenced with a search for [lbi.com] on the google.com site:
The same is also true for a “site:” operator search, which should only return pages from the “lbi.com” domain:
A search for [lbi] shows the expected results, including the correct ‘lbi.com’ homepage, so this is definitely included in the index:
The Leo Baeck Institute website (lbi.org) has no such error, showing that there is not a plain switch of site home pages:
We’ve dropped Google a line and will post further updates here when we hear any news back from them…
Update: Once we highlighted this, Google’s own John Mueller provided a response in the comments below, and within 24 hours the result for [lbi.com] has now been changed to display the expected results, with an LBi.com title, snippet and sitelinks appearing at the top of the page. We would like to extend our thanks to Google for ensuring a swift resolution.
Jack Weir says:
August 11, 2010
So the simple fact that the Leo Baeck Institute used to own lbi.com never occurred to you? Doesn’t show much “insight and creativity”.
John Mueller says:
August 11, 2010
Hi! I’m a webmaster trends analyst at Google and saw @ianmacfarlane’s tweet about this. It looks like something old from a previous version of the domain got stuck on our side. In general, the best solution for something like this is to submit a reconsideration request or to post in our help forum. At any rate, I hope that this will be clearing up in the near future
.
Simon Howland says:
August 11, 2010
Hi Jack, that’s correct, the domain has changed hands. We considered that this could be a bug. As noted above, that really is the only other explanation. However, with the main URL and site links going to one domain and the green display URL going to another does not seem this way – particularly with the cache date being so recent.
Simon Howland says:
August 11, 2010
Hi John, thanks for your comment. We’ve submitted a re-consideration request so hopefully the results will soon return to their natural order
franz says:
August 12, 2010
hi, there is nothing, really nothing that would indicate a manual editing of the search results.
am i thinking correct that the home page of lbi.com was for a long time lbi.com/en/ and that lbi.com actually did a HTTP 302 redirect to lbi.com/en/ which was changed some time ago, so that HTTP lbi.com/en/ now makes an HTTP 302 to lbi.com? just a wild guess.
changing the redirect to from lbi.com/en/ to lbi.com to an HTTP 301 permanent redirect should do the trick.
also fix your expires date which point to Thu, 19 Nov 1981 08:52:00 GMT.
and your robots.txt is wrong.
a specialized googlebot directive overwrite the * directive, so all the generall * rules do not apply to googlebot. please sign up for a google webmaster tools account and validate your robots.txt.
oh yeah, there are some other seo issues with the site, please ask your next trusted SEO for help.
and no, google does not edit the organi results, it does not scale.
Simon Howland says:
August 12, 2010
Hi Franz,
Firstly, take a look at this video of Matt Cutts himself explaining cases where Google “takes action” to manipulate search results: http://www.youtube.com/watch?v=hky-tXyAcqA&sns=em. Google can and do manually edit search results, of course this will only happen in a small minority of cases as you say due to issues of scale.
A redirect of any sort is a highly unlikely reason for a link, snippet and sitelinks to be directed to a different domain to the display URL. If anything, the whole result would be incorrect, no?
So, as I have said above, this is either a mechanical error, which John has suggested is the case, or this is a manual edit gone wrong, which I have suggested is certainly a possible case in my opinion. Feel free to disagree.
As for your other points, we are currently rolling out a site refresh, so many thanks for your observations and watch this space.
franz says:
August 12, 2010
hi,
first
http://web.archive.org/web/*/http://www.lbi.com/
archive.org tries to make snappshots of the startpage and archives them.
shows that http://www.lbi.com == http://www.lbi.org until 2006
second
the lastest available snappshot shows
http://web.archive.org/web/20100812095158/http://www.lbi.com/en/
http://www.lbi.com/en/ ???
as the start page
this means there was some kind of redirect from http://www.lbi.com/ to http://www.lbi.com/en/
so basically http://www.lbi.com/ was a non page.
due to the lack of a start page google compiled site links for the old http://www.lbi.com that it knew about and never changed them.
now http://www.lbi.com/en/ gets depreciated, redirect HTTP 302 to http://www.lbi.com/, but HTTP 302 is the wrong redirect which basically states “http://www.lbi.com/en/” is the right address, http://www.lbi.com/ is just a temporal stop, don’T bother.
so http://www.lbi.com/ is still mostly a non page. you have to make http://www.lbi.com/ your start page and you have to tell google that the old start page is no longer valid.
just change the redirect to HTTP 301, maybe reset the laso mod date of all pages, check your robots.txt.
additionally recover some link juice as your old start page lbi.com/en/ currently gets a sh*tload of backlinks http://www.google.com/search?q=site:http://www.lbi.com/en/&hl=all&pws=0 which now gets wasted.
you can also do nothing, as this issue will resolve over time (at least as soon as you add new content to the startpage and point some links to it), but with the HTTP 301 fixes it will happen faster.
Jeff Parks says:
August 12, 2010
Is Alexa also manually editing results? That seems unlikely.
http://www.alexa.com/siteinfo/lbi.com
Simon Howland says:
August 12, 2010
Hi Franz,
I did indeed review the Internet archive records and the prior configuration of lbi.com. I don’t deny that one of the possibilities is that Google has indeed made an error, and that switching domains, pre-existing link graphs and temporary redirects are complex structures which can be difficult even for the most technologically advanced search engine to deal with.
It is for this very reason that I displayed a screenshot of the lbi.com homepage being returned for a search on google.com for [lbi]. Google, it would appear, is aware of the page at the root domain. Something which after two years should no longer be an issue.
Being in SEO as yourself, you’ll appreciate why I’m considering all possibilities and keeping an open mind. As you say, “When Google compiled site links” they were for the previous version of the site. Which brings into question how manual the process of compiling this listing was, the argument for which I have outlined above, along with the screenshots to annotate my question.
I appreciate your expansion of the issues involved, as it adds to the discussion. Whether this is a case of human editing, discreet from the more usual process of automated SERPS, or a machine error is only known to Google. In any case, an interesting result, worthy of discussion.
Simon Howland says:
August 12, 2010
Hi Jeff, thanks for highlighting this – yes, unlikely. There would be no reason for Alexa to do this. I don’t know how the internals of Alexa are setup to update the listing data, or what processes they have in place to deal with this type of situation, so I can’t be completely sure about the cause of this.