Front to back

The pieces came together today at the DIS 2008 conference - or at least closer together. So far, human-computer interaction has focussed on things like cognition, efficiency and matching existing work practice. But maybe we've got that pretty much that stuff sorted out - at least from a researcher's point of view. Our curiosity is leading us towards a new frontier: how interactive technology can address the social and emotional aspects of our lives.

Matt Jones from Swansea University talked about the StoryBank project. The team has placed a digital screen in the centre of a rural Indian village to help people share knowledge in the form of stories. Participants can create a simple story on their mobile phones, made of still photos and audio, then "gift" it to the StoryBank screen. They can apply simple tags to the story using icons: "kids", "farming", "health" humour"... Other villagers can discover stories floating past on the screen, and view them or download them onto their own phones. The villagers have limited literacy, an extroverted culture and a strong tradition of sharing knowledge through storytelling. They have taken to the screen enthusiastically.

Eric Paulos, from Intel Research, talked about "Objects of wonderment". He asked us to consider how technology can create a sense of wonder, rather than just being a practical problem solving tool. He also asked us to rethink our understanding of mobiles phones and consider them as "public urban processors" - cheap, ubiquitous, connectible modules of computing power attached to humans that move them around the city. This situation allows amazing phenomena to emerge - when it can be harnessed.

The objects of wonderment project has provided a tool kit to enable people to re-use old mobile phones and extend them to do new things. The example: the Hullabaloo - a phone in a box, attached to a loudspeaker. Whenever it detects a new bluetooth ID drifting past (another phone and its owner) it assigns a new bird call sound clip to that person and plays the bird call. Regular passers-by start to be able to identify their own bird call - and that of other regulars.

John Williamson from the University of Glasgow, demonstrated a fun way to browse and explore a large digital photo collection. You literally shake the screen of his hand-held Flutter interface to bring up a new selection of photos, then grab a pleasing photo with the stylus and shake that to release other similar photos.

Factors for designing social and emotional experiences

The conversation after a number of talks touched on questions like "Who would want this?", "Where would this be worth doing?" And compiling ideas from many talks, I ended up with this short list of factors to consider when designing social and emotional experiences. For social experiences:

• Culture: Are you designing for a culture which values the group or individuality more highly? Consider, also, attitudes to family - so different between say, Singapore and the UK.
• Individual differences: Different people have different tastes and behaviours. A given emotional experience is unlikely to appeal to everyone (and why should it?)
• Physical proximity: Is the experience designed to work over a wide distance, or to help people communicate and collaborate better when they are standing right next to one another? If people are together - why? Are they there to collaborate or join in community or entertainment activities, or are they forced together by circumstance? On a crowded train, people use inteactive devices like ipods to separate themselves from the crowd and create much-needed personal space. But at a festival or market, people have deliberately come along to interact with each other.

For emotional experiences:

• Physicality: Physical movement, be it whole-body or just smaller movements, is often playful and can make an experience joyful.
• Where the emotion is: Should the system seek to identify the users emotion, and do something with that "knowledge"? Or should it simply act as a medium for gathering, storing and sending new kinds of material that will produce an emotional responses in the users (yesterday's sensecam story is a great example of this second approach).

Identifying all the variables involved in creating social and emotional experiences is going to take plenty more effort and experimentation. But everyone at the conference accepted that the user-centred design approach, involving users throughout the design process, is the only way to learn. For these cutting edge designers and researchers, UCD is a given.

The third wave

Bill Gaver, from Goldsmith's in London, summed up what's happening. A third paradigm for HCI is emerging (though I'm not yet clear what the first two were). This new paradigm defines interaction design as "situated meaning making" - not only improving efficiency, or matching user needs better, but also adding the emotional and social ingredients to make an experience worthwhile.

Some of Bill's tips:

• Design situations and resources, not tools. In other words, challenge people to think in new ways and give them what they need to make progress.
• Make the world interesting, not the system.
• Interaction can be in the mind. There's not always a need for bells, buttons and sliders if the interface can make you think about the world in a new way.
• Welcome ambiguity and a variety of interpretations, don't view them as error or risk. People have always interpreted and appropriated systems - however rigorously they are designed. This new wave of HCI acknowledges and welcomes it.

To see what he means, take a look at this video (6 mins) of Bill's latest project being evaluated by its users. It's great fun, and fascinating.

And is this approach commercially applicable? Sometimes.

• Interesting interaction can be used to attract attention.
• Devices that create emotion or promote reflection might be desirable. I'd buy a SenseCam.
• We know that emotional connections create brand loyalty. Many Apple users have a strong emotional connection to their computers and the Apple brand. A little of that comes from playful user experience.

But is it art?

It all seems perilously close to art. And surely there are serious limits for this new kind of HCI. Would you prefer to know that the pilot has full control of the plane, or that he has emotional relationship with it, and can interpret its signals to him in a range of ways? Bill points to some intriguing examples:

• The Shared Space project has found that removing road markings and street furniture, to make an intersection ambiguous actually makes people think differently and take more care there.
• And maybe an engaged pilot, who finds his aeroplane eternally interesting is safer than a bored one, or one who has little interaction with his plane until the autopilot suddenly fails.

And how will the new HCI be branded? HCI 3.0? Situational HCI? Now that's the really hard part...

Some great presentations at DIS 2008 carried on the themes of social and emotional interaction.

Maria Håkansson and Lalya Gaye from the Viktoria Institute in Goteborg, Sweden talked about their "context camera." It's a digital stills camera that applies effects to the pictures based on sound and motion that occured as the photo was taken.

Marcus Foth from Queensland University presented Cityflocks - a social navigation tool with a difference. The mobile-phone based system to allow people to write and read restaurant reviews, but it also allowed people wanting information to actually contact a local Cityflocks user and ask for restaurant advice directly - via text message or even a voice call. The designers compare it to asking someone in the street for directions. To me it sounds like a mixture of Zagat and real-time Yahoo answers.

The results: people didn't like the voice call mechanism - too synchronous and intrusive. The text message approach worked well, but it took a couple of days to get answers so it was better for people who were planning ahead.

Microsoft Sensecam research

But my favourite talk of the day was about some recent research undertaken by Manchester Metropolitain university and the BBC using the Microsoft Sensecam.

Microsoft research invented the Sensecam in 1999. It's a light-weight digital camera that you wear around your neck. It takes pictures automatically, when it senses changes in light, heat or motion.You can also set it to just take photos on a regular clock. The photos are 640x480 resolution - and each one is just a rough-and-ready snapshot, taken automatically. You end up with a with a huge mass of photos which you can play back as a timelapse film of your day.

This sounds odd and pointless. And when the researchers gave five sensecams to regular folk, they weren't sure they were going to see anything very exciting.

But the results they got back amazed them. People really connected with medium. They selected unexpected favourite photos. One wrote dialogue to represent the conversation that had been happening at the time of the picture. Another set his timelapse to music - to make an absolutely entrancing 3-minute film. One participant was overjoyed to capture one of those moments when you just wish you had a camera - he caught his girlfriend feeding a dog biscuit to the dog, and eating a dog biscuit herself!

Here are some of the reasons why having a Sensecam could be amazing...

• Imagine seeing all the things in your day you didn't notice, and getting a chance to take a fresh look at how you spend your time.
• Imagine seeing a friend's or partner's day played back to you in a couple of minutes
• Imagine seeing a timelapse film of what your child did all day
• Imagine putting a sensecam on your dog - or attaching it to a kite
• Imagine running your own timelapse day alongside your partner's so you can see what each of you was doing at each moment as the day progressed
• Imagine reviewing days in the life of a deceased loved one

Nokia's lifeblog, and other life-blogging approaches, have already hinted at some of these experiences. But Sensecam makes the whole process close to automatic, and provides a perspective which is close to your own, but not your own. The result is remarkable.

Microsoft don't appear to have plans to manufacture the Sensecam for consumer use yet. I'm looking forward to the day when they do.

Last day of DIS 2008 tomorrow. But I've seen more stuff already than I can blog about!

"There are certain deficiencies of computers right now," said one of the speakers at the DIS 2008 conference  today. And this first day of conference highlighted what researchers are doing to address those deficiencies.

Some solutions are more production-ready than others. I expected DIS 2008 to be "out there" and it is.

Deficiency 1: Interacting with computers feels unnatural.

Researchers are exploring speech and gesture to make interaction more like "everyday life".

Edward Tse, from the university of Calgary, presented some great examples of speech and gesture interaction around a large digital table to allow people to collaborate on affinity sorting notes and images. I would dearly love to try this out on a design project.

Mixed reality allows us to interact with computers by manipulating objects, and our own bodies, in the physical world.

Adrian Cheok from The National University of Singapore's Mixed Reality Lab demonstrated a range of mixed reality experiences, including "human pacman" where players arrayed with cameras, VR headsets, GPS and wireless data connections played pacman by physically running around an area of the city.

Deficiency 2: Computers demand too much attention.

The demands that media and information technology make on our attention every day have been shown to cause stress, mistakes and accidents.

Daniel Robins, from Microsoft research, demonstrated some ideas for how to make a smart phone interface less demanding of our attention.

He makes use of three attention-saving approaches:

1. Glanceable interface: This lets you soak up information when you have a moment to glance at it - without pressing any buttons at all. Daniel proposes dividing the screen into tiles, each of which surfaces a key piece of information. So not just "email" but "3 unread emails"; not just "appointments" but "Appointments at 11:00, 12:00 and 15:00 today.
2. Muscle memory: This enables people to interact with the phone without looking, using "muscle memory". Many of us can touch type, so we experience muscle memory every day. By dividing the screen into 9 "tiles" you can map to the 9 number buttons on the keypad. This means that people can select an option by knowing which position it sits in. They can just press a number and don't need to look at the screen until the information they need is already displayed.
3. Peeking: This allows users to view information quickly and briefly with minimal navigation. By holding down the corresponding key, users can temporarily display more detail about one chunk of information. Let go, and the display drops back to the home screen - showing all nine information "tiles".

Deficiency 3: Computers can't communicate emotion very well.

Software, computers and modern life can cause feelings of isolation. We use technology to combat that: an email or Skype call to a distant loved one can help. But these technologies are limited and that can limit the enjoyment of staying in touch. They let us share information, but are much less good at letting us share experience.

Emotion and enjoyment come via touch, taste, physical motion, what we wear, subtle expressions - many things that current internet technology has no mechanism for communicating.

So Adrian Cheok from Singapore's Mixed Reality Laboratory has invented:

• The poultry internet, to let you stroke your family chicken remotely (a family chicken is a fairly common pet in South East Asia). You stroke a soft dummy chicken, and the real chicken wears a fluffy "haptic jacket" which simulates the stroking on its body. Experiments show that the chickens love it.
• Huggy Pyjamas, to let guilty parents still away at bedtime, hug their children remotely. The parent strokes a small device (something like a key ring in future releases) and air-filled actuator's in the child's pyjamas simulate the parent's hug. The parent can send a signal to change the colour of a badge or patch on the pyjama's too.
• Age invaders, to let grandparents, parents and grandchildren play phsyically together. Some players play over the net, others physically move their bodies on a giant, digital board.



Cheok's team also note that many of us can empathise more effectively with living things that artificial things. (They give the example of real flowers versus plastic flowers: which would you prefer to receive form a loved one?) So to boost the empathy we feel when confronted with information by a computer, they have pioneered the idea of empathetic, living media: computer displays made out of living things.

The Babbage Cabbage experiment lets you tend a garden of six real cabbages that change colour based on data feeds that you select. By injecting an acid or alkaline solution into each cabbage, the cabbages can be made to go from green to purple over a number of days, while staying alive and healthy. You can make the different cabbages respond to all kinds of information from four categories: personal, family, society or environment. Some examples information about your own energy usage, or the environmental impact of your travel behaviour, or how much you are communicating with a loved one via text messages. The team have reproduced the idea with transgenic, glowing bacteria and fish. Next they might try squid, or a display that attracts ants to different areas to make patterns.



Research is not complete yet, but some evidence suggests that people really do feel more empathy when it's a living thing that is displaying ambient data.

More from DIS 2008 tomorrow.

An article on the BBC website recently gave readers a chance to comment on why they didn't have a mobile phone (about 14% of British people don't have them). Some of the mobile "refuseniks" highlight a well-known issue: attention and multitasking...

"Mobiles are like needy children, always wanting attention. I wanted to cut out the stress."

The issue, dubbed "continuous partial attention" by Linda Stone, has been discussed for many years now. Linda points out that continuously staying on alert for new information from a range of sources simultaneously is actually bad for your health - mental and physical. She anticipates a new trend where we will select technologies that protect us from too much "noise" and allow us to focus on quality experiences, relationships and information.

A very entertaining article on TheAtlantic.com predicts a dramatic change in the way use out attention: a multitasking crash, followed by an attention deficit recession. The author, Walter Kern, discusses recent psychological research...

"Researchers asked a group of 20-somethings to sort index cards in two trials, once in silence and once while simultaneously listening for specific tones in a series of randomly presented sounds. The subjects’ brains coped with the additional task by shifting responsibility from the hippocampus—which stores and recalls information—to the striatum, which takes care of rote, repetitive activities. Thanks to this switch, the subjects managed to sort the cards just as well with the musical distraction—but they had a much harder time remembering what, exactly, they’d been sorting once the experiment was over.

Even worse, certain studies find that multitasking boosts the level of stress-related hormones such as cortisol and adrenaline and wears down our systems through biochemical friction, prematurely aging us. In the short term, the confusion, fatigue, and chaos merely hamper our ability to focus and analyze, but in the long term, they may cause it to atrophy."

Social tools consultant and writer Stowe Boyd disagrees. He suggests that human consciousness can handle multiple attention demands simultaneously, and get a lot of joy and satisfaction out of it. He points out that we've been doing for as long as we've been humans...

"One Eye on the flint we are knapping, and one eye scanning the savannah for predators, chatting the whole time."

Pay attention to you

I can only conclude that everyone's capacity for and attitude to continuous partial attention is different. Some people are very social, and enjoy flitting between conversations at the virtual cocktail party. Others prefer deep, zen-like focus on a single task. And some of us will mix and match depending on what the day brings.

But whatever your natural ability, it's important to make sure that you use your attention the way your want to - don't let the technology control you.

Watch yourself for the next week. Are you spreading your attention so thinly that you end up stressed and exhausted? Are you actually getting anything done? If not try this:

• Reduce the number of communication channels you use. Do you really need twitter and facebook and a blog and e-mail Skype and MSN Messenger and text messages...? Really? Choose a few channels and let people know that those are your preferred ones.
• Set chunks of time when you disconnect from some or all of your channels. For example...

"[...] in recent weeks prominent bloggers have been choosing to disconnect. Ken Camp turned off Twitter for a week, and didn't miss it. Robert Scoble announced last week that his blogging would be slowing down, because he was going to work on getting back into shape. Mark Evans has chosen to not check BlackBerry mail after 6 PM." (Saunderslog)

• Use technologies to help you filter the noise and free your attention. Servcies like Digg or Amatomu will select the best things for you.

Now stop reading this, go find a good book, get comfy and read that for a while instead.

(Thanks to Martin Storey and Simon Johnson for the pointers.)

Thanks, Debre, for pointing out the strategy that Starbucks have been following: cutting costs and downgrading service in a bid to stave off competition from MacDonalds.

From the Guardian...

The troubled coffee chain Starbucks, renowned for its elaborate frappuccinos and mochas, is going back to basics by testing a cut-price brew costing only $1 (51p). Fighting slowing growth at its US stores, the firm is offering an eight-ounce "short" measure of ready-made coffee for a price undercutting fast-food rivals such as McDonald's in a trial at branches in its home city of Seattle.

This looks like an appalling strategy. For the simple reason that it focuses on coffee.

Experience vs Commodity

Starbucks didn't build a business on selling coffee. Coffee is cheap. They built a business on selling a customer experience.

The term they used was a "third place" a restful, aromatic, aesthetically pleasing, sociable space that is neither work nor home. Coffee was the hub of it, but without the surrounding experience, would any of us really consent to pay two pounds a cup?

Apparently, the experience is now all but gone. And hence it's becoming rather difficult to justify the price tag for coffee alone. More from the Guardian...

Schultz recently warned in a leaked internal memo that the brand's charm was in danger of diminishing as it became a mainstream "commodity". He said the sense of theatre had evaporated, thanks to automatic espresso machines, and he complained some stores even no longer had an aroma of fresh ground coffee due to vacuum-sealed packaging.

Innovation pundit Bruce Nussbaum relates his recent experience of walking into a Starbucks in New York City…

I thought […] I’m going to feel like a sausage on an assembly line, waiting, talking to people not paying attention, then waiting again. And for what? A cup of coffee? It was all so transactional. I don’t need Starbucks for that.

CEOs who get it

Apple has been down this road. During their darkest hour, they were producing a bewildering array of uninspiring machines and an ageing, unreliable operating system. They wanted to compete with the frequently drab IBM PC clones, and in so doing seemed to forget their "think different" mantra. It was Steve Jobs, much to everyone's surprise, who put Apple back on track by helping it deliver a unique user experience again - starting with the brightly coloured iMac.

It seems that Starbucks has also smelled the coffee. Howard Schultz, the newly appointed chief executive is the guy who built Starbucks up in the first place. He does seem to know what he's doing - note that he talks about the "sense of theatre".

So why the $1 cup of coffee? No matter home much pressure Starbucks is under, competing on price to "lure" customers back in doesn't make sense for an experience-based company. Better to refurbish the stores, refocus the staff and refresh the coffee. Then let word of mouth bring customer back for what they were always buying anyway: the experience.