Customer satisfaction: the lead metric (2)

In an earlier post I pondered meaningful approaches to measuring customer satisfaction.

I’ve bumped into a couple of items about measurement which I enjoyed. Nothing conclusive yet.

  1. Hilarious article by Joel Spolsky about how management consultants get big businesses to pay money for nothing, because they persuade those businesses to pay attention to the wrong metrics.
  2. Interesting article from Fast Company about how business folk like to measure and prove, but designers like to peer into customer needs and desires, then have ideas based on those insights and push into the unknown.

I’ve also found out a bit more about the approach used by the telecoms operator I mentioned last time. It seems they picked a metric to tackle: churn was the first. Then they used several qualitative research methods to understand the factors affecting the metric. And then they rapid prototyped some solutions in a small area of their customer base to see what impact they could have.

Again, it seems to come down to…

  • Business metrics to identify problem areas and measure success
  • Qual research to discover how to improve the metric


I just discovered you can use CTRL-Tab to tab through windows within an application. Things like workbooks within Excel. Or tabbed browser windows within Firefox.

This is fiddly, but I’m pleased that there is at least some kind of shortcut.

You knew this one? Oh ok. It was news to me. And I found it just by a slip of the finger on the keyboard!

Another one that people often don’t know about is F2. In Excel or Powerpoint or Explorer, hitting F2 allows you to edit the text inside something. Be it a cell, a shape or a filename.

What does this have to do with user experience? While I work on a posting about it, why not try using the shortcuts for a while and see what you think.

Customer satisfaction: the lead metric

I ended up in a discussion with someone working for a mobile phone operator the other week. They were talking about performance metrics:

“We’re measuring ARPU (average revenue per user), signups and leavers,” he told me. “Oh – and customer satisfaction. But we don’t really worry about that much. And anyway it’s very hard to measure.”


Why do customers use services on their phones and generate revenue? Because those services are well designed and offer a good user experience.

Why do customers leave? Because they are unhappy with the customer experience (including price, in these days where mobile operators are now actually trying to retain their customers).

Why do customers sign up? Because the company has a good reputation, and offers a good customer experience.

So surely measuring customer satisfaction would be a good idea. If customer satisfaction goes up, you can be certain that all the other metrics will follow.

Trouble is, customer satisfaction is hard to measure in any meaningful way. Surveys are glib, and your customers often can’t imagine what would make them more satisfied. “Yeah, it’s fine” is not the same for your business as “I love this thing!” But surveys can’t tell you how to get from one to the other.

Measuring “symptoms” like ARPU and churn might be the best answer. But it worries me. What gets measured gets done. So measure customer exerience, and your organisation will focus on it. Measuring symptoms could make your organisation focus on the symptoms and start trying to tinker with them, without really understanding why customers are behaving as they are.

Measuring what people do with a service as it stands tells you something about the popularity and performance of the service over time. But it doesn’t tell you why the service is or is not popular. You can speculate about why. But you’ll never really be able to deduce user intentions from the data. And without undestanding user intentions, you’ll struggle to predict what would make your design radically better. Which leaves you in the realm of tinkering, rather than changing the market.

I suppose what I’m concluding is that measurement isn’t a great tool for innovation. It’s a good tool for proving the value of innovation – or the need for it. But if you want to create services that customers love, you’ll need to use techniques that help you understand their desires, needs and abilites. Qualitative, ethnographic techniques are the way to go there.

World *usability* day

Jared Spool put his finger on (or close to) something that was bothering me.

“Usability” doesn’t improve the world. In fact, it doesn’t change a thing. […] It’s the practice of design where the change comes from.

I must say, I have a problem with where the word “usability” has ended up.

Usability practitioners have a reputation for being pedantic, and focussed on listing problems rather than proposing solutions. Some of the reputation is deserved. I know many who are quite happy to write down generalised recommendations to usability problems in longhand, rather than engaging with the real complexity of the problem and drawing the answer. Just writing “recomendation: make the submit button easier to find” does not constitute a solution at all in my book.

The term usability was officially defined as “efficiency, effectiveness and satisfaction […]” in the ISO document, and the definition referred to “specific users doing specific tasks in specific contexts of use”. Once you understand those words, that covers a lot of ground and shows the very best intentions.

But somehow, usability has ended up, in the minds of many, as being just the “ease of use” part. “Can I find the button?”
The terms User experience and customer experience seem to have won the day. Those terms have a breadth to them that works. They seem to capture something important: that with great product and services, the whole is greater than the sum of the parts. In some cases, you can have bad parts, and still have a great user experience.
An example. There’s a lovely gastro-pub just down the road. It is decorated with cream wood panelling and amazing pink wallpaper. A warm inviting glow reaches out as you pass in the damp November street. The food is delicious, the wines well selected. Wood fires crackle happily and charming, chunky stoves. The staff are jovial, the customers are happy and all seem to be in the mood for a chat. But when you come to leave, the door need to be pushed, even though it has a pull handle!

Focussing on the handle seems churlish – even pointless. The landlord of the pub has achieved a customer experience which is truly delightful. The unusable handle is insignificant in comparison. At its worst, usability has become associated with people who would obsess about the handle, and miss out on joy of the amazing pub that surrounds it.
But ultimately, as Jared Spool says, you need a verb – design. Sitting around listing problems is at best a first step. Working out viable solutions is better. That’s why I’m proud to have been a small part of the MakingLifeEasy blog, set up for World Usability Day. There’s a lot of intelligent ideas in there for improving the designs that get listed – even the good designs. (Admittedly, the next step would be to include drawings too).  If we’re going to talk about usability for a day, we should be as constructive as possible.
Despite my reservations, I admit that considering usability can still bring joy. I went to visit my sister on the weekend before World Usability Day. She said, “I’m too stupid to know what all those buttons on the microwave do.”

My wife (UX fanatic too) set her straight. “If you can’t use it, it’s the product’s fault, not yours. That’s what usability is all about.”

“Well,” said my sister. “That is a refreshing perspective.”

Interaction design is not an island

Recently, someone called me up from a large media corportion asking for a training course in website interaction design. They told me they wanted to know what to put above the fold, and how to make a good navbar. They said they could use this best practice knowledge to win battles with editorial and marketing staff about how microsites should be designed.

Fair enough. That stuff is worth knowing, even if you can’t always get it down into cast iron guidelines (“it depends”). But, I pointed out, knowing today’s take on best practice interaction design will only have limited impact on the final user experience. That’s because user experience emerges from the efforts of so many different parts of the organisation. The developers need to deliver that the users need, the front end coders need to create standards-compliant XHTML, the editorial staff need to communicate meaningful and comprehensible information, the bsuiness folk need to think about how it will make money. And someone in charge (producer/product manager/e-commerce manager/…) needs to drive the whole thing in a meaninfgul conceptual direction. A good user experience is the product of the organisation behind it. And everyone needs to be pointing in the same direction to achieve it.

Getting everyone pointing in the same direction is hard. UCD requires people to do things in new and potentially unsettling ways. There are two and a half ways to persuade people and organisations to adopt UCD, that I know of:

  1. ROI models, and case studies
  2. Running usability tests
  3. (Tactical web analytics)

So I suggested a UCD training course to my prospective client. One which would tell the designers how to “mount a campaign” that will convince an organisation of the value of the work, and steer it to a successful conclusion.Yes, came the reply, but. Really we just want to know how long the page can scroll. And whether orizontal navbars are ok. The site concepts come to us fully formed from the editorial team and we just paint them up. We’re not going to be able to change the product development process, and it’s not really our job to try.

I proposed an interaction design course, which will be fun and useful. But I couldn’t help thinking: if you really want to see great user experiences going out the door with your name on them, you have to take some responsbility. Good interaction designers should have the quality of the total user experience foremost in their minds. They are the people who should “get it” and lead the rest of the team towards a user-centred approach.

Recently, I attended SPARK UX, a symposium about UX and software architecture hosted by Microsoft at Half Moon Bay in California. In one of the many chats that happened during the two days, Jeff Patton told me about the term “single ringable neck”. It’s a term that sums up cutely with the idea that ultimately you need one person to be responsible for a project – or in this case, the user experience. And that person should, in my book, be the interaction designer.

Responsibility and power need to go hand in hand. Power is earned through effort, results and a willingnessto accept responsibility. Allan Cooper, in “The Inmates are running the asylum” talks about “skin in the game” – the idea that if you’re not seen to be the one who will suffer if things that go wrong, people on the development team won’t take you seriously.

Ultimately, I was hoping to persuade my prospective client to put some skin in the game, and begin the process of raising the quality of interaction design at his organisation. We’ll start slowly with them – thinking about detailed interaction design.

You have to start somewhere…