Real customer satisfaction cannot be measured by "un-real" customers

I've been reading a lot lately about the results of a satisfaction study conducted of the top search engines - and how Google took second fiddle to Yahoo this year.

This is a bold statement. But who's making it? Knowing enough about our industry, I'm confident enough to say that it isn't real visitors to the sites of these two online megaliths (by real visitors, I am referring to people who are on the site, at their own free will, and being asked to provide feedback within the context of a real experience). Perhaps the figures were derived from panel research or offline interviews. While there is nothing wrong with this - they are effective forms of research and certainly have their place - one thing is for certain, they do not provide a true measurement of the actual voice of customer.

Why not? Simple.

Participants in these research vehicles are not real customers within a real experience. Theirs does not reflect a spontaneous and unscripted experience, and therefore cannot be substituted for real people trying to accomplish a real purpose of visit. As Duff Anderson (Vice President Research and Development at iPerceptions) pointed out in a previous post, our research shows that the way task-based or scripted visitors rate satisfaction is not the same as the way actual customers in real situations rate it. What is important to them and how satisfied they are is very different.

Here's an interesting analogy. NEC has decided to start wiring its labs with high-tech surveillance equipment to snatch up water-cooler talk and break-room chit chat, in the hopes that the real nuggets of brilliance will emerge when its employees are speaking in an unguarded fashion. The reasoning is clear: when you compel someone to furnish an answer in an uncomfortable setting, the truth will often be distorted. However, when you allow someone to speak in the context of a spontaneous experience, real insight emerges.

It is easy to understand, therefore, why these much-publicized Google/Yahoo findings struggle to align with the latest comScore numbers. If Google has really been "resting on its laurels" over the past year, then why does comScore's data show that Google' share of searches has actually grown from 46% in July 2006 to 55% in July 2007? More and more searchers are flocking to Google, with most of the growth coming at the expense of Yahoo. The disparity in stock performance between Google and Yahoo has also been widely-documented.

It's certainly an eye-opener.

The Internet has allowed us to dialog with consumers in the context of their real experiences. For the first time, people can speak for themselves. I believe that the Google/Yahoo results would look quite different if these were real people in the context of a real experience.