Let’s say you’re planning to migrate to a new high-redundancy CS core network—what’s the first challenge that comes to mind? For many of you, it might be network architecture. The structure of modern telco networks is increasingly complex, and any new network rollout is going to be much more involved as a result. To wit, you’ve got to make sure that you’re setting up the core network protocols correctly (in accordance with the 3GPP standards), that your network can interconnect successfully with other networks, that it has functioning access layers for end users devices, and much more.
As the average telco network—and thus telco testing—gets more and more complex, the value of automation is becoming increasingly obvious. Whether you’re a device tester looking down a mountain of use cases requiring verification or a product manager desperate to improve your time-to-market, the ability to run through hundreds of use cases per day with the press of a button has to have an obvious appeal. And yet, the way we discuss test automation for telecoms often feels a little abstract. Broader discussions of how and why testers should automate might leave readers wondering what, exactly, an automated test environment would look and feel like.
Raise your hand if a subscriber has ever complained to your company because she wasn’t getting the Quality of Service (QoS) she was paying for. We’re guessing that a lot of readers will have raised their hands. Sometimes, complaints like this arise because of forces beyond the control of the backend offices that actually handle QoS (if the subscriber lives on the border of your LTE coverage, for instance, and simply can’t get a strong enough signal to achieve the level of service she’s paying for), but there are certainly times when these issues arise because of an error in the provisioning process.
Up to this point, the most common use cases for the IoT have continued to be industrial: tracking pallets as they move through a warehouse, monitoring inventory levels, analyzing machine usage, etc. But with the advent of 5G, consumer applications for this technology are going to become much more prevalent—especially as the speed of over-the-air data transfers become comparable to sending the same information over a wire. To handle this inevitable influx, the 3GPP has been setting standards and protocols for NB-IoT, with a focus on Low Power Applications (LPWA) over licensed spectrum bands.
It might seem somewhat paradoxical, but the better your LTE coverage gets, the harder it becomes to test something like SRVCC. Verizon, for instance, has about 93% LTE coverage in the United States (according to a recent OpenSignal report), which means that at most there’s only 7% of the country where a handover from LTE to a 2G/3G circuit switched call bearer network would be appropriate. In all likelihood, that limited geographical area is going to be nowhere near the offices of any of Verizon’s testing teams—which means that testing these handovers manually will require testers to potentially drive for hundreds of miles in search of areas that aren’t covered by their existing VoLTE service.
For the best possible results, your test lab needs to make use of the absolute latest in cutting edge technology. Because this is the arena in which you test changes and additions to your network—including new connected devices like the internet of things (IoT)—you need to find a way to replicate existing network conditions as closely as possible. Otherwise, you might test out new service offerings or network adjustments in a lab setting only to find that your service doesn’t work correctly under real life conditions—which could result in costly delays, outages, and potential subscriber churn.
One of the biggest concerns we hear from telco operators looking to automate their tests is the relative ease or difficulty of test case scripting. Telecoms aren’t typically staffed with a huge number of technically proficient developers, and a complex testcase scripting language would therefore make it difficult to leverage non-technical personnel in the testing process. This is an eminently reasonable concern. After all, the fewer people there are in your organization who can understand the tests being performed, the more likely you are to find yourself saddled with a testing silo—which could potentially lead to slower bug fixes and poor alignment between testing and other functions.
At telco operators around the world, test engineers and operations managers are experiencing a bit of a conundrum. With networks growing in complexity by the day and manual testing getting more time consuming than ever, the need for automation is becoming obvious. At the same time, not all automation is created equal, and testers need to make sure they’re equipped with the right tools for the challenges that await them as modern networks continue to evolve.
Since the introduction of LTE in 2009, most telco operators have had to maintain three networks in parallel—2G, 3G, and LTE—with smooth interworking between the three of them, sometimes even in the course of a single phone call. Not only that, but operators need to offer their customers seamless connections with a host of other legacy systems, from ISDN to POTS. Each time the larger telco landscape gets more complex (which happens practically by the day), the presence of existing legacy systems amplifies that complexity—making life increasingly difficult for the testers tasked with verifying network functionality.
At most telco operators, end-to-end tests are the dream. Given that each new network update or service offering potentially requires dozens of testcases verifying conformance, acceptance, functionality, and performance, true end-to-end tests that validate the entire functioning of the network from the end-user’s perspective are daunting and often difficult to accomplish (especially if you’re testing by hand). As such, your average network tester might balk at the idea that end-to-end testing wasn’t enough to ensure high network quality—considering that going end-to-end is no mean feat in itself.