T&M: The Testing Time

The telecom arena has become fiercely competitive over the last decade as
markets around the world have liberalized and new entrants have emerged to
challenge the incumbent service providers. In the battle to win customer
loyalty, companies now-more than ever before-have to focus on the quality
and efficiency of their products and services offered and, consequently, testing
assumes critical importance. There is an urgent need to get products or services
to market more quickly than at anytime in the past, in order to gain a
competitive edge. This pressure leads to tight testing schedules to meet the
short deployment timescales.

A few weeks, or even a day's, delay in launching a new or improved service
can mean the difference between achieving profitability or worse, making a loss.
It is imperative, however, that the requirement for rapid testing should not
adversely affect quality of neither the new nor the existing services. Despite
the shorter time for testing, companies must ensure that when solutions are
released, they work exactly as specified with all the potential glitches
detected and corrected during early testing phases.

Ten
years ago, testing was not a high priority for service providers. Customers had
few options to look elsewhere if they were unhappy with the service provided by
incumbent PTTs. Today's landscape is radically different and telecom companies
are generally far more attuned to actively searching out the highest quality
testing solutions and those that most closely meet their specific requirements.
The emergence of automated testing provides a significant advantage to telecom
equipment manufacturers (TEM), independent service vendors (ISV), and service
providers (SP) alike. Many of whom have, in the past, had to rely exclusively on
the traditional manual testing processes.

The Automated Test Process

Ideally, these companies require comprehensive testing in which each feature
is tested with various inputs and constraints, with as many iterations to the
process as needed to assess the product fully and reliably. Due to conflicting
time-to-market pressures, the testing teams cannot possibly examine all
interactions between all components manually in a timely and cost-effective way.
This is where automation really excels.

There has been a rapid proliferation of tools that address the detection of
errors early in the development process and the testing of software
automatically. Of these, probably the most common of automated tools is the 'record/playback'
type. Typically, the tester navigates through all combinations of menu choices,
dialog box choices, buttons, etc. in the application under test and has these
actions 'recorded' and the results logged by the tool. The 'recording'
is in a scripting language that is interpretable by the testing tool. When
required, the application can then be re-tested by just 'playing back' the
'recorded' actions, and comparing the logging results to check effects of
the changes.

The key benefit of automated testing-particularly in today's highly
competitive telecom world-is that it accelerates the speed of the testing
process and thereby reduces time-to-market and boosts return on investment (RoI)
for the company. In  addition, through better test coverage and accuracy,
it delivers higher-quality products and services.

Automated testing does not suffer from the cumulative effects of minor
distractions and delays, which could be the case with manual testing. Automation
also allows tests to be executed 24 hours a day, seven days a week.
Organizations that currently use a single eight-hour shift for their manual test
cycles will immediately realize a 67 percent reduction in elapsed time by
migrating to18 hour-a-day automated testing. If weekends and holidays are
factored in, the benefits become even more attractive.

To get a feel for the impact of these improvements, consider that a human
tester works about 2,000 hours a year out of a potential of about 8,760 hours.
If the organisation concerned can fully exploit 24x7 testing then the
compression of the test cycle could reach over 75 percent. These improvements
can lead not just to faster time-to-market for products and solutions but also
potentially to an increase in the testing window.

As a result, they make more time available for companies to carry out a
broader variety of tests and, ultimately, to ensure the best possible product
quality. Automation also reduces the cost of testing. This is due to fewer
resources being needed. Manual testing requires engineers familiar with the
design of the products being tested, the characteristics of likely or known

problem areas, and the sequence of events that must be followed to recreate
particular test scenarios. These engineers are expensive both in terms of their
salaries and the cost of dedicating them to the mundane and repetitive tasks of
testing. Reducing these costs is a major advantage of implementing automation.

Another important benefit is the ability to support regression testing, which
tests compatibility between versions of software and additional products or
services. The rapid rate of technological change in today's dynamic telecom
arena means that products or services can potentially be out of date six months
after launch. The global nature of business often means that minor modifications
or adjustments are required to ensure a product or service is suitable for
launch in a broad range of different markets. As a result, equipment
manufacturers and service providers are more inclined to adjust, modify, or
enhance service or product offerings in a bid to achieve a competitive edge
across geographies. Regression testing, supported by the automated approach,
makes this a lot easier.

What to Look For

So, what makes an attractive proposition for those companies looking at
implementing automated testing solutions? Ideally, a vendor that specialises in
the telecom sector and has a proven track record in large testing projects-with
core expertise in business support systems (BSS) and operational support systems
(OSS), and is capable of automating complete testing processes and offering a
complete end-to-end testing capability.

Service providers who carry out testing themselves find that it is very
expensive to implement and maintain. The cyclical nature of releases results in
under-utilization of costly testing facilities and possibly prolonged idle time
of expensive resources assigned exclusively to testing projects. What would
improve cost efficiencies is a contractor who can automate the testing process
end-to-end, provide and maintain test labs, and deliver expertise and resources
on demand to carryout a broad range of important test types-from operational
readiness (ORT); verification, validation, & test (VV&T); integration
testing; and also measuring the performance of networks under various stress
scenarios.

It is vital that the testing solution vendors provide a broad range of
service packages that meet individual and specific requirements. These might
range from fully managed operations to consultancy on the selection and use of
testing tools and techniques. The consultancy team should also make
recommendations regarding the process streams by undertaking business process
re-engineering (BPR) and security of the overall systems under test. This has
become fundamentally important in the modern world of virus attacks. Vendors who
comply with standards like CMM Level 5, ISO 9001, and BS 7799 should be sought,
to ensure the highest testing quality. A full security audit should also be an
important part of the offering, and at the end of the testing process itself,
the selected vendor should provide the customer with a comprehensive project
report.

Ultimately, the success of automated testing is all to do with trust.
Customer satisfaction long into the future should be the goal for both parties
in the partnership.

Anthony D'Souzacorporate
head (systems and Network Testing Business Unit) Mahindra British Telecom