Proceed to WirelessDevNet Home Page
Publications, e-books, and more! Community Tutorials Store Downloads, tools, & Freebies! IT Career Center News Home
newnav.gif

Newsletters
EMail Address:



   Content
  - Articles
  - Columns
  - Training
  - Library
  - Glossary
 
   Career Center
  - Career Center Home
  - View Jobs
  - Post A Job
  - Resumes/CVs
  - Resource Center
 
   Marketplace
  - Marketplace Home
  - Software Products
  - Wireless Market Data
  - Technical Books
 
   News
  - Daily News
  - Submit News
  - Events Calendar
  - Unsubscribe
  - Delivery Options
 
   Community
  - Discussion Boards
  - Mailing List
  - Mailing List Archives
 
   About Us
  - About WirelessDevNet
  - Wireless Source Disks
  - Partners
  - About MindSites Group
  - Advertising Information
 

Symbian DevZone - "TRY" - Automated Applications Testing for Symbian OS phones

by Richard Bloor

Testing is a critical part of the software development process and often a well thought out application concept can stumble due to poor software testing. Testing is however a resource intensive task. Does Mobile Innovation’s Try offers developers the opportunity to automate their Symbian OS software testing and not only save time but improve the quality of their software? This week we take a look at TRY’s capabilities. - Printer Friendly Version



Last year we looked at the first commercial testing tool for Symbian OS phones, TestQuest Pro’s Interface for Symbian OS. This testing tool is currently targeted at hardware manufacturers as it runs tests through a phone’s hardware. Now Mobile Innovation (http://www.mobileinnovation.co.uk) have introduced the first commercial testing suite targeted at applications developers, although as Gunnar Larsén, Products & Marketing Manager, points out TRY can equally be used in testing new Symbian OS phones.




Jonathan Sulenski and Matt Millar formed Mobile Innovation in May 2002 when they undertook a management buyout of a joint venture platform development being undertaken by Symbian and a third party. Mobile Innovation’s web site describes the project enigmatically as “one of the most successful Symbian platform implementations to date.”

TRY was developed by Mobile Innovation to address the needs of projects they were undertaking. “TRY has been an integral part of our internal testing processes for the last 2 years,” said Gunnar Larsén. “We have found that TRY has allowed us to detect defects earlier and more reliably, saving us money and meaning we can deliver a higher quality product to our customers. It has become an essential tool for us. Because of this we felt that there was an opportunity to bring similar benefits to other Symbian developers and as a result we decided to productize TRY and bring it to market.”

TRY currently runs on Series 60 and Series 80 with a version for UIQ planned for release at the beginning of August. The software runs on both the emulators and phones, it is self-contained including an editor for creating and maintaining scripts (although script creation functionality for Series 60 is part of the August release). Scripts can also be created and edited using any text editor capable of saving Unicode files, which includes the basic Windows Notepad editor.

Externally edited scripts are made available to TRY by either copying them to the TRY directory on an emulator or Series 80 device or sending them via infrared or Bluetooth to a Series 60 phone where they are automatically copied to the TRY directory when opened.




The scripting language is straightforward, similar in command structure and syntax to BASIC. This means that test scripts can be edited and maintained by test staff who may not be familiar with Symbian C++ or Java, leaving developers free to concentrate on rectifying defects and working on further development.
StartApplication "Calendar"; Open the calendar application
Pause 2; Pause to allow app to open
PressCBA 0; Press left softkey to open menu
Pause 2;
PressCBA 0; Press left softkey to open day
Pause 2;
PressCBA 0; Press left softkey to open menu
Pause 2;
PressKey 17; Scroll down to New Entry
Pause 2;
PressCBA 0; Select New Entry
Pause 2;
PressCBA 0; Select Meeting
Pause 2;
PressKey 56; Press the 8 key (tuv)
PressKey 55; Press the 7 key (pqrs)
PressKey 57; Press the 9 key (wxyz)
PressKey 48; Press the 0 key (space)
PressKey 54; Press the 6 key (mno)
PressKey 51; Press the 3 key (def)
Pause 2;
PressKey 51; Press the 3 key (def)
PressKey 56; Press the 8 Key (tuv)
Pause 2;
PressCBA 1; Select Done
Pause 2;
PressCBA 1; Select Back

Sample TRY Script to create a Series 60 Calendar entry.

Sample Script in action

In addition to the core capability of mimicking user input to a Symbian OS phone TRY has a number of features which make it more than a simple action scripting language. These features include FTP file transfer, data input and output, high- level data creation commands, logging and system manipulation.

TRY has the ability to trigger the execution of a script resident on a device by sending an SMS. One potential shortcoming of this feature is that the test results are recorded on the phone and the phone would need to be returned to the office for the results to be evaluated. This is where TRY’s FTP features should be invaluable. TRY is able to send and receive files via FTP which means that the results of remotely executed tests can be returned to a central server for review without having to bring the phone back.

The ability to read and write data to simple flat files means that tests can be driven by bulk test data, increasing the number of tests that can be run and also meaning that the number of scripts required can be minimized. While TRY does not include high level commands for extracting data from a record it has sufficient text string find and manipulation functions to allow fixed length or demarcated data, such as that in a CSV file, to be processed. Coupled with the FTP transfer it also means that even remote tests can be run with different test data sets as testing requirement’s demand.

Obviously running tests requires some form of verification that the software has performed as expected. Here TRY offers two methods of validation. The first allows the tester to specify the comparison of two screenshots and get confirmation on whether they are identical or not. As some screens could contain information that varies independently of the test, such as time and date displays, TRY allows a mask to be defined so that sections of the screen can be excluded from the comparison.

The other verification method allows specific text, in one of the standard fonts, to be found within a screen. If there had not been a calendar entry in the time slot where the new appointment was added, in the example script, “Open” would not have been on the menu. As a result the script would have failed to create a new record. By adding the following code the script could have checked for the presence of “Open” on the menu and omitted the down keystroke if it was not there.
FindText n_x_coordinate, n_y_coordinate, “Open”;
If (n_x_coordinate!=-1 && n_y_coordinate!=-1){
        Pause 2;
        PressKey 17;
        }

This is clearly a powerful feature as it allows the creation of general purpose scripts, ones that react to the results of entering and processing test data. TRY also allows for the capture of screen shots and it’s logging capability allows the results of screen or text verification to be recorded.

Another interesting feature worthy of a mention is the command that creates standard application entries such as calendar entries, tasks, contacts, messages and even Symbian Word, or Sheet documents. Mobile Innovation claims that this command can, for example, create a contact directory of 2000 records in less than 10 seconds. This function also clearly makes scripts similar to the example somewhat unnecessary!

The evaluation version of TRY is limited in a number of ways; it is not possible to create scripts with more than 10 commands, there is no ability to call subscripts, only one create item command (such as messages and contact) can be included in a script and the loop commands are disabled. Therefore it is not possible to give a comprehensive analysis of how well TRY performs, however within the limits it certainly ran the short scripts created.

Productizing an internal tool is not always a straightforward task, invariably much of the knowledge of the tool is with the people who use it day to day and gathering that information is often not easy. TRY unfortunately is a good illustration of this problem as there are a number of places where the documentation does not fully or accurately explain the workings of TRY. This means that the initial impressions of the product are not entirely favorable, as in some cases the behavior is not intuitive. Mobile Innovations support team were able to quickly resolve most of these issues. Mobile Innovation is aware of this issue and has plans to update the manual later this year.

TRY is available in three versions designed to cover a range of uses from “basic” applications testing through to use in new device development. The versions are: Developer, which provides the run time versions for both emulator and device. Access to TRY FAQs and discussion threads as well as periodic product updates (with an annual maintenance change of 20%) are also provided.

Enhanced, which in addition to the Developer package also provides access to source code and enhanced support response.

Corporate, which is targeted at device manufacturers with further enhancement to the support service and the ability to run the product with development kits. All TRY licensees provide for unlimited numbers of users within the licensing organization and in the Corporate version this covers sub-contractors as well.

TRY however is not going to be a solution for all software developers because of its price. The basic version costs $35,000 and as such is unlikely to appeal to smaller development shops building Symbian OS software. “Working out an appropriate licensing model which suits everyone is not an easy task, so for this initial launch we have focused on larger users,” said Gunnar. “While we have no specific plans we will be looking at the options for creating a basic license to make TRY accessible to the small software house as we certainly believe TRY would be of benefit to them as well.”

While the scripting possibilities of the evaluation version of TRY are limited they strongly suggest TRY has all the capabilities necessary to allow comprehensive automated testing to be developed and run on Symbian OS software, with the possible exception of games and highly media orientated applications. The scripting language is straightforward and should not need specialist developer support. However there is clearly some work that needs to be done on the supporting documentation.

Digia will be bringing their testing tool to market later this year and it will be interesting to see how it’s functionality compares to TRY and what pricing structure they adopt. TestQuest are also understood to have a software version of their testing tool in the wings.

You can download the evaluation version of TRY from Mobile Innovations web site (http://www.mobileinnovation.co.uk/downloads.php) along with the product brochures, licensing information and ROI calculation spreadsheets.




About the WDN Symbian Editor, Richard Bloor:
Richard Bloor is a freelance writer and editor with 18 years experience in the IT industry as a developer, analyst and latterly Project Manager with a particularly focus on software testing. Richard has been involved with the Symbian OS since 1995 and has been writing about it for the last 3 years.

Richard is also an associate with System Architecture consultancy Equinox of Wellington, New Zealand.

Richard can be reached at symbian@wirelessdevnet.com.

Symbian DevZone Home

Sponsors

Search

Eliminate irrelevant hits with our industry-specific search engine!









Wireless Developer Network - A MindSites Group Trade Community
Copyright© 2000-2010 MindSites Group / Privacy Policy
Send Comments to:
feedback@wirelessdevnet.com