Front to back

From BBC News Online...

'[Virgin trains] denied reports in the Daily Telegraph about automatic doors opening unexpectedly or locking people in.

Problems with disabled toilets on the hi-tech tilting trains however were down to people not using the automatic doors properly, a spokesman said.

"There's a button to close the door and another with a key symbol on it which locks the door and flashes when the door closes," said David Ewart, communications manager with Virgin.

"It's pretty clear what you have to do. We've even got signs in Braille," he added.

Rail watchdog group Passenger Focus, however, said there might be a need for Virgin to have clearer instructions for customers.'

What a mad world we live in where people design electronic toilet door buttons... and get it wrong! And some poor soul ends up getting caught with their trousers down on the 9:13 from Birmingham New Street. Surely a manual door would have been ok. Then a nice handle which affords sliding would have been crystal clear.

But now we've got watchdog groups to make sure that the toilet doors get improved. And reporters to write stories about the whole business. And bloggers to blog about it. And Virgin has to refit the doors, and create improved signage.

Think how much time and money could have been saved if the engineer had just designed the door right, and usability tested it in the first place. Then the whole lot of us could go off and do something more meaningful. (There is a lesson here about hasty roll-outs of interactive products and the true time and cost of reaching customer satisfaction. But I'm not going to spell it out any further).
PS. If anyone 'goes' on a pendolino, do try and get to the 'bottom' of this one for me.

This follows on from my two previous posts on this subject.

I think I've got somewhere with this. It has been an interesting ride. The questions boil down to:

• How can you measure the quality of your customer experience, across multiple channels (web, store, call centre, WAP)?
• Is measuring symptoms of customer experience, rather than the experience itself "dangerous"?
• Are those metrics the only data you need to help you deliver a good customer experience?
• Why would you measure customer experience related factors anyway?

And here are the answers.

How can you measure the quality of your customer experience, across multiple channels (web, store, call centre, WAP)?
Observing users, rather than gathering their opinions, is one of the cornerstones of usability and UCD methodology. It's effective. So to measure customer experience, it makes sense to follow the same approach: observe what users do, rather than asking them what they think.

In the case of the telecoms operator I first spoke about, I think their decision to measure ARPU and churn is the right one. The observed action of loyal, satisfied customers is to stick with them and spend more. Customers vote with their wallets.

And conversely, customer satisfaction surveys are not a good measurement - those are opinion based, and opinion can't be trusted.

I discovered a different telecoms operator that measures customer dissatisfaction on a monthly basis. This is an odd half-way meausre. If it's done by survey, it's not much good. If it's done by measuring actual customer complaints across all channels, it's better. But it's not great. Even if you got customer complaints to zero, that doesn't really tell you if your customers are happy - just that they are not complaining any more.

If we concentrate on the web channel for a moment, there are some unique methods of measuring customer satisfaction. Tools like Relevant View and WebIQ allow you to intercept customers on your site, track them around, and ask them whether they are getting what they want at key moments. These tools do yield some wonderful information, but for a truly multi-channel business they are not enough. If you're working across multiple channels you need metrics that are relevant across all channels. ARPU and churn, for a telecoms operator, fit the bill.

Is measuring symptoms of customer experience, rather than the experience itself "dangerous"?
The possible issue here was that measuring a "symptom" of the customer experience (eg. ARPU) might lead the business to focus on the revenue itself, rather than the cause of the revenue (ie. customer experience).

There are three counter examples, and that indicates to me that there's no problem here.

• The original telco I've been talking about set about a program of study and experimentation to find and prove the factors that were affecting ARPU, and change them. They didn't get too hung up on the metric, but successfully went looking for the causes.
• Nokia meaures the sales of each mobile device it launches, and pays each product team's bonus based on that. Nokia makes very good phones, and you can see why. The engineers know that if they make something people like, it will sell well, and they will benefit personally.
• Apple do the same.

Are those metrics the only data you need to help you deliver a good customer experience?
No. 'Course not. But lots of businesses have made the mistake of thinking they are.

To design a new product or service, you need to understand the motivations, abilities and desires of the target user group and deliver something that addresses those needs. UCD is great at that: start with ethnography or contextual enquiry, and later when you're in the thick of concept and detailed design, get more user input from usability testing.

If you're fixing an existing customer experience, you need to understand where it's broken. Mystery shopping, diary studies, expert reviews, call log and search log analysis, web analytics and usability tests can all help.

Why would you measure customer experience-related factors anyway?
Two key reasons, that I can see.

1. Measuring hard numbers and linking them to the quality of customer experience is a great way of demonstrating the value of CX initiatives to the business. (Proof based businesses)

2. Using those numebrs as the basis for team reward, like Apple and Nokia do, is a great way to drive certain kinds of positive behaviours in the business. Supply the incentive and watch the business gravitate towards good UCD practice. You won't have to force designers to design for user needs or conduct usability tests. When their bonusses are riding on customer satisfaction, they'll do everything they can to engage with their customers during design. (Faith based businesses)

More about faith and proof in a future posting.

A few years ago, my wife bought an electric blanket for our bed. She did well- it's wonderful thing and it keeps us snug.

But the usability of the control modules is so dire that it's actually funny. Take a look...

Note...

• The cryptic red circle icon drifting between the on-light and the left hand button. Function: unknown.
• The green left and right arrows beside the other button. How does that button allow you to go left and right? It only has a normal press action.
• And the three way switch, that means that you can't just turn the blanket on without really looking at the control to make sure you're not accidentally switching into timer mode.

Still - let's be thankful for the twisty temperature knob. It's much better than a button-based effort.

Of course, this UI is the product of a company that wanted to save money on design. And we bought the cheapest blanket, so we really weren't expecting the controls to be any good. It keeps us warm, and that is the most important part of the user experience.

But the issue is this. If they wanted to make the controls cheap, they should just have kept them simple. An on-off switch, a light and twisty knob would have been great. Instead they incurred extra engineering and manufacturing costs by adding timer features that no-one can use and that no-one really needs, especially not on the budget blanket.

How did it end up so?

The key fact: Simplicty is harder to achieve than complexity.

Engineers want to add features because it's fun and challenging and because the features make sense on paper.

Marketers want to add features because they think it makes the product look more compelling in-store.

So put them in a room together and you're not going to get anyone crossing features off the list.

Some solutions:

• Incentivise the engineers on sales. The more of their products that get sold, the bigger their bonusses. Nokia and Apple do it like this. By all reports, it works frighteningly well.
• Get the marketing department to start marketing the virtues of simplicity. It's a competitive angle. For example: SimpleHuman.

In an earlier post I pondered meaningful approaches to measuring customer satisfaction.

I've bumped into a couple of items about measurement which I enjoyed. Nothing conclusive yet.

1. Hilarious article by Joel Spolsky about how management consultants get big businesses to pay money for nothing, because they persuade those businesses to pay attention to the wrong metrics.
2. Interesting article from Fast Company about how business folk like to measure and prove, but designers like to peer into customer needs and desires, then have ideas based on those insights and push into the unknown.

I've also found out a bit more about the approach used by the telecoms operator I mentioned last time. It seems they picked a metric to tackle: churn was the first. Then they used several qualitative research methods to understand the factors affecting the metric. And then they rapid prototyped some solutions in a small area of their customer base to see what impact they could have.

Again, it seems to come down to...

• Business metrics to identify problem areas and measure success
• Qual research to discover how to improve the metric

I ended up in a discussion with someone working for a mobile phone operator the other week. They were talking about performance metrics:

"We're measuring ARPU (average revenue per user), signups and leavers," he told me. "Oh - and customer satisfaction. But we don't really worry about that much. And anyway it's very hard to measure."

Interesting.

Why do customers use services on their phones and generate revenue? Because those services are well designed and offer a good user experience.

Why do customers leave? Because they are unhappy with the customer experience (including price, in these days where mobile operators are now actually trying to retain their customers).

Why do customers sign up? Because the company has a good reputation, and offers a good customer experience.

So surely measuring customer satisfaction would be a good idea. If customer satisfaction goes up, you can be certain that all the other metrics will follow.

Trouble is, customer satisfaction is hard to measure in any meaningful way. Surveys are glib, and your customers often can't imagine what would make them more satisfied. "Yeah, it's fine" is not the same for your business as "I love this thing!" But surveys can't tell you how to get from one to the other.

Measuring "symptoms" like ARPU and churn might be the best answer. But it worries me. What gets measured gets done. So measure customer exerience, and your organisation will focus on it. Measuring symptoms could make your organisation focus on the symptoms and start trying to tinker with them, without really understanding why customers are behaving as they are.

Measuring what people do with a service as it stands tells you something about the popularity and performance of the service over time. But it doesn't tell you why the service is or is not popular. You can speculate about why. But you'll never really be able to deduce user intentions from the data. And without undestanding user intentions, you'll struggle to predict what would make your design radically better. Which leaves you in the realm of tinkering, rather than changing the market.

I suppose what I'm concluding is that measurement isn't a great tool for innovation. It's a good tool for proving the value of innovation - or the need for it. But if you want to create services that customers love, you'll need to use techniques that help you understand their desires, needs and abilites. Qualitative, ethnographic techniques are the way to go there.

Recently, someone called me up from a large media corportion asking for a training course in website interaction design. They told me they wanted to know what to put above the fold, and how to make a good navbar. They said they could use this best practice knowledge to win battles with editorial and marketing staff about how microsites should be designed.

Fair enough. That stuff is worth knowing, even if you can't always get it down into cast iron guidelines ("it depends"). But, I pointed out, knowing today's take on best practice interaction design will only have limited impact on the final user experience. That's because user experience emerges from the efforts of so many different parts of the organisation. The developers need to deliver that the users need, the front end coders need to create standards-compliant XHTML, the editorial staff need to communicate meaningful and comprehensible information, the bsuiness folk need to think about how it will make money. And someone in charge (producer/product manager/e-commerce manager/...) needs to drive the whole thing in a meaninfgul conceptual direction. A good user experience is the product of the organisation behind it. And everyone needs to be pointing in the same direction to achieve it.

Getting everyone pointing in the same direction is hard. UCD requires people to do things in new and potentially unsettling ways. There are two and a half ways to persuade people and organisations to adopt UCD, that I know of:

1. ROI models, and case studies
2. Running usability tests
3. (Tactical web analytics)

So I suggested a UCD training course to my prospective client. One which would tell the designers how to "mount a campaign" that will convince an organisation of the value of the work, and steer it to a successful conclusion.Yes, came the reply, but. Really we just want to know how long the page can scroll. And whether orizontal navbars are ok. The site concepts come to us fully formed from the editorial team and we just paint them up. We're not going to be able to change the product development process, and it's not really our job to try.

I proposed an interaction design course, which will be fun and useful. But I couldn't help thinking: if you really want to see great user experiences going out the door with your name on them, you have to take some responsbility. Good interaction designers should have the quality of the total user experience foremost in their minds. They are the people who should "get it" and lead the rest of the team towards a user-centred approach.

Recently, I attended SPARK UX, a symposium about UX and software architecture hosted by Microsoft at Half Moon Bay in California. In one of the many chats that happened during the two days, Jeff Patton told me about the term "single ringable neck". It's a term that sums up cutely with the idea that ultimately you need one person to be responsible for a project - or in this case, the user experience. And that person should, in my book, be the interaction designer.

Responsibility and power need to go hand in hand. Power is earned through effort, results and a willingnessto accept responsibility. Allan Cooper, in "The Inmates are running the asylum" talks about "skin in the game" - the idea that if you're not seen to be the one who will suffer if things that go wrong, people on the development team won't take you seriously.

Ultimately, I was hoping to persuade my prospective client to put some skin in the game, and begin the process of raising the quality of interaction design at his organisation. We'll start slowly with them - thinking about detailed interaction design.

You have to start somewhere...