The complexity of today's smartphones is a world away from early mobile phones. With increasing sophistication comes the risk that the user will simply get lost in all the options available to them. It is not enough to be working to prevent the interface gridlock brought about by a mountain of features. Platform developers need to show that they have processes in place to ensure that, as more feature are added, smartphones remain usable and above all enjoyable.

It is against this background that UIQ Technology has started to reveal information on UIQ Technology Usability Metrics (UTUM) the process used to gather usability information as part of the UIQ platform development process. Richard Bloor caught up with Laurent Mauvais, Interaction Architect and Mats Hellman, Head of Interaction Design to find out more.

Richard: Why is UIQ making its usability methodology public?

Laurent: From the time we started, in 1998, usability has been an important area for us at UIQ Technology. As the UIQ platform delivers a range of rich functions such as messaging, email, PIM, and many more, it is clear that usability is something we need to get right. Without good usability the extensive functionality delivered on UIQ phones would not be easily accessible to the user.

Our focus in our early years was very much on business orientated phones. While that certainly did not make usability easy, it was not as complex as the requirements for the mid-tier, volume lifestyle phones UIQ is now starting to appear on. For a mass market audience usability becomes much more complex, as there is a large range of user expectations to fulfill.

We are publicizing UTUM so we can show that UIQ Technology puts usability at the heart of our platform development work and that we have a robust way to deal with the diversity of mass market usability requirements. By doing this, when we talk about usability, we can show it's built into the UIQ platform on the basis of a solid methodology, not just some vague notion of making it easy for users to use a phone.

Richard: You are making a lot of the fact that UTUM is different from the standard practices used in the industry. How is it different?

Mats: Traditional usability testing relies very heavily on metric orientated around simple measures, such as how many clicks it takes to do something or how long something takes to achieve.

With UTUM we are really concentrating on how satisfied the user is. We are interested in their overall experience. This is because good usability is not only about how quickly or easily something can be done, the process should also be fulfilling for the user. Ultimately we want the user of a UIQ phone to enjoy using it.

So we believe UTUM is different because it looks at the whole user experience, not just clicks.

Richard: So how does a typical UTUM usability session work?

Laurent: We would typically start by asking the user to complete a questionnaire. This would gather information about how they use their existing phone. From this we can determine the features they are interested in. Then we work with the user to choose the tests they should run.

Testing then uses an ethnographic approach. This is because we are not only interested in the quantitative measures of usability, how many clicks it took to do something and how long it took, but we are also interested in the qualitative information we can gather. Testing is therefore very collaborative. We work with the user, we listen to them; we don't dictate what the tester should do. In many ways it's a very Swedish approach.

We are not asking simply "is this feature easy to use". Rather we are trying to understand the complete user experience. For this reason we often perform tests outside our labs, in places where our testers feel comfortable.

Many traditional usability testers may view this approach as being overly complex and time consuming, but to us it is important. This is because the UTUM approach engages the user, it asks them about things that are relevant to them, not some arbitrary set of tests. Because the tests are relevant to the user we believe we get more useful information from the process.

There is another practical dimension to this. Understanding the things that are important to the user helps provide a more complete understanding of the results. If we conduct two tests, one with a user who is primarily interested in using their phone for messaging and another who likes using their phone to listen to music, they may provide very different usability scores for a particular feature. Without understanding the difference in interest the results can simply look random. Armed with knowledge of the tester's interests and habits the results can be meaningfully interpreted.

Richard: Does the qualitative approach make the process more time consuming?

Mats: It could do, but we have refined our process so it does not. Typically we can perform a validation test on an interface design with twelve testers in about a week, with another week to collate the results.

Richard: How does UTUM fit in with your technical platform development?

Laurent: UTUM is completely integrated with our development processes. So we use UTUM to test concepts, test interfaces while they are being developed, and validate the final product. It also scales well from dealing with usability of an entire phone, right down to testing a small component within the interface. This means we can use it throughout development, to continually validate the work we do.

Richard: How do you see UTUM contributing to the success of the UIQ platform?

Laurent: We believe that the successful smartphone of the future will be easy and enjoyable to use. If the industry continues adding more features to smartphone platforms without properly considering usability, as many developers are doing, phones will become unbearably complex. For us UTUM is an integral part of our development process, but more importantly it allows us to find and evaluate new ways for the user to interact with their phone; to measure not only whether interactions are simple, but whether they are enjoyable and satisfying to the user.

A full paper on UTUM will be published by UIQ Technology in October, in the mean time you can find more information on UTUM at http://www.uiq.com/utum.html.

UIQ Technology will be showcasing UTUM as part of their presence at the Smartphone Show, Excel Center, London, on 17th and 18th October.