As the average telco network—and thus telco testing—gets more and more complex, the value of automation is becoming increasingly obvious. Whether you’re a device tester looking down a mountain of use cases requiring verification or a product manager desperate to improve your time-to-market, the ability to run through hundreds of use cases per day with the press of a button has to have an obvious appeal. And yet, the way we discuss test automation for telecoms often feels a little abstract. Broader discussions of how and why testers should automate might leave readers wondering what, exactly, an automated test environment would look and feel like.
It might seem somewhat paradoxical, but the better your LTE coverage gets, the harder it becomes to test something like SRVCC. Verizon, for instance, has about 93% LTE coverage in the United States (according to a recent OpenSignal report), which means that at most there’s only 7% of the country where a handover from LTE to a 2G/3G circuit switched call bearer network would be appropriate. In all likelihood, that limited geographical area is going to be nowhere near the offices of any of Verizon’s testing teams—which means that testing these handovers manually will require testers to potentially drive for hundreds of miles in search of areas that aren’t covered by their existing VoLTE service.
One of the biggest concerns we hear from telco operators looking to automate their tests is the relative ease or difficulty of test case scripting. Telecoms aren’t typically staffed with a huge number of technically proficient developers, and a complex testcase scripting language would therefore make it difficult to leverage non-technical personnel in the testing process. This is an eminently reasonable concern. After all, the fewer people there are in your organization who can understand the tests being performed, the more likely you are to find yourself saddled with a testing silo—which could potentially lead to slower bug fixes and poor alignment between testing and other functions.
At telco operators around the world, test engineers and operations managers are experiencing a bit of a conundrum. With networks growing in complexity by the day and manual testing getting more time consuming than ever, the need for automation is becoming obvious. At the same time, not all automation is created equal, and testers need to make sure they’re equipped with the right tools for the challenges that await them as modern networks continue to evolve.
At most telco operators, end-to-end tests are the dream. Given that each new network update or service offering potentially requires dozens of testcases verifying conformance, acceptance, functionality, and performance, true end-to-end tests that validate the entire functioning of the network from the end-user’s perspective are daunting and often difficult to accomplish (especially if you’re testing by hand). As such, your average network tester might balk at the idea that end-to-end testing wasn’t enough to ensure high network quality—considering that going end-to-end is no mean feat in itself.
MVNOs (mobile virtual network operators) sometimes get a bad rap. This isn’t entirely without reason: A recent report found that MVNO download speeds across the U.S. are 23% worse than those of their host networks on average. While download speeds for MVNO often could reach the same heights as their host networks, they tended to offer those speeds less consistently, leading to lower “consistent quality scores” in terms of meeting minimum speeds and staying below allowable levels of jitter, latency, and packet loss.
Lots of articles have argued that the first ever internet of things (IoT) device was a soda machine at Carnegie Mellon that was connected to the early internet in the 1980s. The machine was famously automated to let users on the local network see how recently the machine had been filled (and thus how cold the soda was). It’s a great story—but one device does not an internet make. In other words, what we in the modern era understand as the IoT isn’t really about individual devices; it’s about a tremendous volume of different devices all working together in tandem.
In the US right now, Verizon and T-Mobile are battling it out for who can provide the most extensive LTE coverage. And as of OpenSignal’s 2018 report, the two were in a statistical tie—each was able to provide an LTE signal 93.7% of the time. These are historic highs for the companies, and objectively impressive numbers, but that still means that they’re failing to provide an LTE signal fully one-twentieth of the time. As such, SRVCC (Single Radio Voice Call Continuity) would still be critical to providing high quality of service (QoS). Why? Because right now it’s the standard solution for ensuring mid-call handovers from all-IP LTE networks to circuit-switched call bearer 2G/3G networks.
When it comes to banking and financial services functionality, “almost” doesn’t count. From your end users’ perspective, they’re either able to complete their desired action—whether that’s checking their balance, transferring funds, or setting up automatic payments—or they’re not. They’re not going to spend a lot of time looking for workarounds, they’re simply going to register their displeasure by choosing a new app or a new service provider.
If there’s one thing your customers are certainly paying attention to, it’s their bills at the end of the month. We’ve all read about customers who have accidentally been charged enormous sums above and beyond the correct amount, and we’ve all no doubt recoiled in horror at the PR snafus that inevitably result for the telco operator in question. Needless to say, any steps that a business can take to avoid this kind of error are worth their weight in image consultations and damage control.