One area of Symbian OS development that has been far from neglected this year is testing. SymbianOne is aware of at least four commercial testing tools for Symbian OS and a fifth is about to enter the market.

The main driving force behind the diversity of testing tools for Symbian OS has been the demands for platform and handset testing, but the available solutions are no less suited to testing third party applications. For handset manufacturers testing is perhaps the most costly and time consuming aspect of their development. The majority of testing has in the past been undertaken by hand and the best early automation was to use robots to physically press device keys, a costly exercise in its self. It certainly comes as a surprise to those outside the industry to know that if a manufacturer wanted to perform cold weather tests on a device they actually sit a group of testers in a cold room – testing could hardly get less appealing.

Intuwave already has considerable experience with Symbian OS. The company was founded by ex-Symbian personal and a component of its m-Network product, m-Router was chosen by Symbian to replace the in-house and proprietary connectivity subsystem in Symbian OS V6.0. Today m-Router ships with most Symbian OS phones.

m-Test builds on the capabilities of m-Router to provide a PC based application that can drive a connected device. m-Test provides an interface which displays a representation of the device keyboard and screen, when the user types on the PC keyboard, or, for touch screen devices, clicks on the screen, m-Test sends these commands to the phone, driving the device. m-Test then capture the resulting screen display and presents it on the PC. m-Test can also capture the screen image to analyze the effect of the test actions. In addition, because m-Router is so well integrated into Symbian OS, m-Test has the ability to directly interact with the device to, for example, transfer files to the device or gather low level information such as memory utilization and device status.

m-Test was developed so that it was inherently scriptable, by exposing a set of APIs. So while m-Test does not contain its own scripting capability other tools, both those specifically designed for testing and PC programming languages, can drive tests and analyze the outcomes with m-Test.

“When we started looking at test solutions we recognized that there were already a number of tools in the market that provided the ability to script and run tests,” says Andrew. “So rather than create a complete, vertically integrated, solution, which simply reinvented the wheel, we chose to create m-Test to integrate with existing test solutions.”

And this is where Mercury comes in.

Mercury are currently the world wide number one supplier of application test automation products. They supply a range of tools which address both functional application testing and various aspects of performance testing, as well as management applications that assist with planning. defining, executing and tracking test execution. While Mercury's tools provide comprehensive support for a wide range of traditional PC and Server based testing requirements they have historically not addressed the wireless market.

"We identified two demands in the wireless world, one for application and handset testing and another for monitoring wireless services, their performance and availability." says Errol. “Our business model means we work with a number of external experts in the wireless field, and it was one of these, Infuse Consulting, who introduced us to Intuwave. We had been looking at a number of solutions for wireless testing at the time and our evaluation found m-Test to be the best option."

The introduction took place about a year ago and almost immediately the solution was introduce to one of Mercury's existing wireless handset customers.

m-Test will now become an integral part of Mercury’s Business Technology Optimization (BTO) technology, which provides tools for application management, application delivery (which includes testing) and IT governance. Within the suite of application delivery tools m-Test interfaces with QuickTest Professional (QTP) to create and execute test scripts, and can take advantage of the capabilities of Mercury TestDirector to create and manage the tests, Mercury Business Process Testing to define tests at a process rather than application level and Application Delivery Dashboard to provide overview information on test progress.

QTP allows tests to be created simply through a record and run process, but also uses a Visual Basic style scripting language that allows test actions and verifications to be modified from recorded scripts or created from scratch.

Many of the features of the m-Test, QTP combination are similar to other currently available testing tools, it can drive any user orientated command into a device over a Bluetooth connection, or indeed any available connection between the phone and a PC, and then capture and analyze the resulting screens. Unfortunately m-Test has not been able to overcome one of the inherent limitations of Symbian OS device testing. With PC testing tools it is possible to identify specific components on the screen and query their properties directly. Because Symbian OS draws the screen as a bit map, checking the output is limited to various forms of pattern matching. m-Test provides matching routines to test that screen shots are equal at a pixel level as well as text recognition capabilities to confirm the presence of specific data on screen.

Where m-Test has a significant advantage over existing testing products is in being able to drive two or more devices simultaneously. It is therefore possible to create a test in which one phone creates and send an MMS and then confirm that the MMS is received by another phone.

The other significant advantage of m-Test is the relationship with Mercury. “Because Mercury has such a dominant market position,” says Andrew, “many of the handset manufacturers are using it to plan and report on testing activity and create scripts for emulator based testing. Switching to another scripting language and tool infrastructure for handset testing would involve a huge cost in re-engineering their existing test assets. By integrating with Mercury m-Test allows the manufacturers to leverage their existing testing to test handsets, which not only saves money but allows them to automate handset testing much faster.”

Andrew sees m-Test being applicable to at least three broad areas of mobile device and software creation, manufacturers, ISVs and Operators. “As phones get more complex the Operators face a increasing workload to complete their network testing before approving a phone for use on their network,” says Andrew. “Even without the capabilities of Mercury driving m-Test it can help operator testing because it is usually faster for their QA staff to drive the phone from a PC keyboard. In addition when they encounter problems they can take a screen shot which makes describing any anomalous behavior far easier.”

Initial the m-Test solution is being rolled out in EMEA (Europe, Middle East and Africa) but Errol notes that there is considerable interest from the American market as well. In the longer term Mercury also has plans to support other mobile operating systems. "Its too early and possibly unwise to tie ourselves into one platform," says Errol, "however before we address other platforms it is our intention to establish a good presence in the Symbian space before broadening the offering."

m-Test combined with Mercury provides one of the most comprehensive solutions for testing Symbian OS devices or application software, supporting testing from initiation through planning, scripting and execution to results reporting. As many of the delays in device launches have, at least anecdotally, been associated with software faults the need for this type of solution is clearly present. While activities to reduce the cost of handset components are an important cornerstone to driving Symbian OS to the middle market, testing solutions could have a more dramatic effect in speeding up both time to market and market acceptance by delivering higher quality device to the consumer.

m-Test is available via Mercury on a per seat license basis. Intuwave currently do not have plans to support any other testing or scripting products.

More information is available from either Intuwave’s web site (www.intuwave.com) or Mercury’s Web site (www.mercury.com).