I ended up in a discussion with someone working for a mobile phone operator the other week. They were talking about performance metrics:
“We’re measuring ARPU (average revenue per user), signups and leavers,” he told me. “Oh – and customer satisfaction. But we don’t really worry about that much. And anyway it’s very hard to measure.”
Interesting.
Why do customers use services on their phones and generate revenue? Because those services are well designed and offer a good user experience.
Why do customers leave? Because they are unhappy with the customer experience (including price, in these days where mobile operators are now actually trying to retain their customers).
Why do customers sign up? Because the company has a good reputation, and offers a good customer experience.
So surely measuring customer satisfaction would be a good idea. If customer satisfaction goes up, you can be certain that all the other metrics will follow.
Trouble is, customer satisfaction is hard to measure in any meaningful way. Surveys are glib, and your customers often can’t imagine what would make them more satisfied. “Yeah, it’s fine” is not the same for your business as “I love this thing!” But surveys can’t tell you how to get from one to the other.
Measuring “symptoms” like ARPU and churn might be the best answer. But it worries me. What gets measured gets done. So measure customer exerience, and your organisation will focus on it. Measuring symptoms could make your organisation focus on the symptoms and start trying to tinker with them, without really understanding why customers are behaving as they are.
Measuring what people do with a service as it stands tells you something about the popularity and performance of the service over time. But it doesn’t tell you why the service is or is not popular. You can speculate about why. But you’ll never really be able to deduce user intentions from the data. And without undestanding user intentions, you’ll struggle to predict what would make your design radically better. Which leaves you in the realm of tinkering, rather than changing the market.
I suppose what I’m concluding is that measurement isn’t a great tool for innovation. It’s a good tool for proving the value of innovation – or the need for it. But if you want to create services that customers love, you’ll need to use techniques that help you understand their desires, needs and abilites. Qualitative, ethnographic techniques are the way to go there.