|
Newsletters
|
|
|
|
|
The WDN Symbian DevZone... Richard Explores The TestQuest Interface for Symbian OS
by Richard Bloor, June 17, 2002
Testing of any software or hardware product can be both a long and
laborious process. Automated tools can take much of the pain out of
repetitive testing. Both Digia and TestQuest have announced testing tools
specifically tailored to the new Symbian OS phones, TestQuest’s product is
the first to be commercial available.
Symbian have in the past used a number of test automation packages. They
found that, while they were useful for testing on OS emulators they were
unable to assist with testing on devices. To address this, Symbian
developed a set of DLLs that allow a Symbian OS device to be controlled, via
software, from a PC. STAT makes use of the extensive and powerful
Symbian OS APIs to mimic user input. However, this only provided
connectivity to the device; it did not provide test tool capabilities.
Symbian became aware of TestQuest and their TestQuest Pro software
through their involvement with a range of mutual partners in the wireless
device and PDA arena. Initial discussions suggested that TestQuest
understood Symbian's business expectations and their customer's
requirements very well and that they would be an excellent partner offering
complementary technologies capable of maximizing Symbian’s connectivity.
This resulted in the development of support for Symbian OS within the
TestQuest Pro product.
TestQuest have been building test products since 1999. Their tool,
TestQuest Pro, is available to test a variety of embedded systems such as
handheld devices, PDAs and mobile phones, Interactive TV, as well as
general computing applications. TestQuest Pro is a complete tool for
automating functional testing. Current customers include hardware
manufacturers such as Palm, Sony and Nokia and applications developers
such as Epocrates and Wireless Knowledge. Symbian have adopted
TestQuest for their own internal testing and I understand that a number of
Symbian Licensees have also purchased the product for use in the
development of new smartphones.
Unlike traditional test products, which are by their nature intrusive as they
interact with the operating system to drive input and capture test results,
TestQuest Pro typically runs the tests at the hardware interface. As a result
the interaction more closely resembles how a user would interact with a
device. In addition, unlike a purely software testing product, TestQuest is
able to test physical attributes of a device like vibration alert, device LEDs
and sound. This approach also means that there is no danger that the
testing itself could effect the operation of the software under test.
TestQuest Pro consists of a PC application and a test harness that
physically connects to the inner workings of the device. When the tests are
executed electrical signals are delivered to the underlying hardware, keypad,
switches, touchscreen etc. To record the outcome of tests the harness
physically interacts with the output hardware, display, buzzers etc.
capturing display images and electrical signals associated with the
activation of other hardware elements.
With the introduction of Symbian OS support, TestQuest Pro is also able to
drive a Symbian OS-based device or the PC emulator for that device via the
serial port. This obviously has a huge advantage in allowing tests to be
undertaken before hardware is available. Similarly for 3rd Party application
developers who may not have access to hardware or, if they do, don't want
the expense of connecting the test harness, using the serial port is a more
suitable option. TestQuest is also developing this capability for other
platforms.
Tests are developed within TestQuest Pro’s “Recorder,” a complete
environment for creating test scripts. Tests can be created using a graphical
"virtual device" that enables the tester to interact with the device under test
from the PC as though the device was being physically operated.
This interaction can be recorded as a test script. These scripts are created
in standard ANSI C complemented with a large set of pre-built functions for
various test actions. Once created the scripts can be edited. Alternatively,
scripts can also be created directly in Recorder. To support the development
of the scripts TestQuest also provides an interpreter and debugger.
TestQuest Pro’s Test Information Manager provides the ability to manage
and execute test suites. When the test is run TestQuest captures the
various outputs and makes a comparison with the expected result.
Software-based testing products are able to do this intelligently, for example,
by being able to check the value of a field regardless of where it is on the
screen by trapping the data as it is sent to be displayed. As TestQuest Pro
captures screen images it uses a different approach. The first level of
checking is simply to compare screen bitmaps, however these can become
difficult to maintain as the UI changes. To overcome this issue, TestQuest
Pro also has a text recognition capability to enable text elements within a
captured screen to be found and verified. The tester is able to train
TestQuest Pro for any font that may be used on the devices using a font
recognition learning wizard. Once a font has been learned, TestQuest Pro
can recognize (and verify) text in that font. This ability simplifies much of
the test maintenance where only the presence of certain text needs to be
verified, rather than the content of either a portion or the entire screen.
TestQuest is also designed to be extensible. It has the ability to integrate
with other testing products both as the test client and server. So an existing
test application can drive TestQuest Pro or TestQuest can control 3rd party
test equipment with a suitable software interface. Data-driven scripts can
be developed using data in Excel spreadsheets, for example.
Symbian are currently defining a detailed strategy to implement TestQuest
within their development process, but the intention is that it will be used for
full system validation of all future Symbian's OS releases and will be
available to all their developers. At present use is on an ad-hoc basis, even
so Symbian have already been able to make gains in several areas. Test
execution times have been reduced by upwards of 50% in smoke and
regression testing. Tests that were taking 3 days effort have, in the best
cases, been reduced to half a day. The ability to run tests unattended
overnight is also contributing to increased testing efficiency, as has
TestQuest Pro’s ability to automate the administration of test logging.
Sanj Prabhakar, Vendor Project Manager at Symbian, sees TestQuest as
adding the next level of functionality above their existing automated testing.
He expects the extra functionality such as bitmap verification, text
recognition, and complete C scripting language will help Symbian, its
licensees, carriers, and 3rd Party application developers to reap the full
benefits of using automated testing.
The rapid implementation of TestQuest Pro has also been possible in part
because of the record facility. Not only does it speed up the development of
scripts it also allows users to acquire the skills needed to use the solution
effectively in a shorter time.
Sanj Prabhakar commented that "We are very excited about the prospects of
TestQuest Pro for the Symbian OS for two major reasons. Firstly, in terms
of the benefits we see it bringing to our Customer's, improving time to
market through faster device testing/development and improved quality
assurance. Secondly, for the assistance that it will bring to our internal
system validation process."
The mobile phone, as has often been commented, is a personal device,
which the owner expects to be available for use when they want. There is a
far lower level of tolerance with software or hardware defects than may be
accepted in a traditional PC. Smartphone users will have been used to
reliable, always-available phones and the fact that the additional complexity
of Smartphones leads to a greater risk of failure will be of no concern to
them. While automated testing does not guarantee quality it does enhance
the hardware or software developers’ ability to ensure that their product has
been subject to a range of extensive and exhaustive tests. TestQuest Pro
addresses the testing requirements for these new devices with facilities
which allow both hardware and software developers to be able to test the
product attributes most relevant to them.
The TestQuest Interface for Symbian OS is available from TestQuest at US
$7500 per license. A TestQuest Pro license is also required. For more
information see their web site - http://www.testquest.com/.
About the "WDN Symbian Guy" Richard Bloor:
Richard Bloor has 16 years experience in the IT industry. His earlier
work was largely in design and development of commercial and
manufacturing systems but more recently has focused on
development and test management of government systems.
Richard Bloor is the Mobile Applications champion at System
Architecture consultancy Equinox of Wellington, New Zealand.
Richard can be reached at rbloor@wirelessdevnet.com.
Symbian DevZone Home
|
|
|
