As the average telco network—and thus telco testing—gets more and more complex, the value of automation is becoming increasingly obvious. Whether you’re a device tester looking down a mountain of use cases requiring verification or a product manager desperate to improve your time-to-market, the ability to run through hundreds of use cases per day with the press of a button has to have an obvious appeal. And yet, the way we discuss test automation for telecoms often feels a little abstract. Broader discussions of how and why testers should automate might leave readers wondering what, exactly, an automated test environment would look and feel like.
One of the biggest concerns we hear from telco operators looking to automate their tests is the relative ease or difficulty of test case scripting. Telecoms aren’t typically staffed with a huge number of technically proficient developers, and a complex testcase scripting language would therefore make it difficult to leverage non-technical personnel in the testing process. This is an eminently reasonable concern. After all, the fewer people there are in your organization who can understand the tests being performed, the more likely you are to find yourself saddled with a testing silo—which could potentially lead to slower bug fixes and poor alignment between testing and other functions.
Since the introduction of LTE in 2009, most telco operators have had to maintain three networks in parallel—2G, 3G, and LTE—with smooth interworking between the three of them, sometimes even in the course of a single phone call. Not only that, but operators need to offer their customers seamless connections with a host of other legacy systems, from ISDN to POTS. Each time the larger telco landscape gets more complex (which happens practically by the day), the presence of existing legacy systems amplifies that complexity—making life increasingly difficult for the testers tasked with verifying network functionality.
As of a few years ago, more than 90% of software testers reported that they were automating between 50 and 100% of their tests. Of the survey respondents who had automated, about a quarter saw ROI immediately, another quarter within 6 months, and another quarter within a year. Fewer than 10% failed to reach ROI. Naturally, the telecom domain is its own beast, and in all likelihood the numbers for automation adoption would look a little bit less robust—but an examination of similar trends adjacent to the telco industry can still be telling for network operators and testers.
Let’s say you’re in a management position at prominent European network provider, and you’re trying to assess network quality. Some of your biggest questions are about testing: How quickly are your regression test suites running after network updates? How frequently do your tests uncover bugs, and how do those bugs get resolved? To gain answers to some of these questions, you contact your test team, who send you the most recent test reports—but you can’t make heads or tails of them, and your questions remain unanswered.
Let’s say that after all these years you’re finally ditching your legacy billing system in favor of something new and shiny. For a while now you’ve been hearing about the OSS/BSS solutions of the future, with their unprecedented ability to integrate with IP networks, and you’ve finally decided to take the plunge. When all is said and done, you’ll have transitioned from a system that was basically designed to tally up voice usage and SMS traffic to spit out appropriate bills, to something that offers your customers real-time tracking of their data usage so that they can manage their behavior accordingly. This has the potential to be a huge win, but there’s just one problem: how do you make sure it’s working?
Often, when we talk about quality of service (QoS) we tend to focus on things like latency, jitter, and packet loss, i.e. the network conditions that users experience in their day-to-day mobile device usage. But back-office processes like provisioning and invoicing can often have just as much of an impact on subscriber happiness in the long term—both because subscribers sometimes have to interact with billing systems and because smoother functioning in the back office often helps network operators to improve service elsewhere. For this reason, savvy telco operations tend to think long and hard about storage and access to subscriber information.
Lots of articles have argued that the first ever internet of things (IoT) device was a soda machine at Carnegie Mellon that was connected to the early internet in the 1980s. The machine was famously automated to let users on the local network see how recently the machine had been filled (and thus how cold the soda was). It’s a great story—but one device does not an internet make. In other words, what we in the modern era understand as the IoT isn’t really about individual devices; it’s about a tremendous volume of different devices all working together in tandem.
Back in 2017, the UK launched its 5G Testbeds and Trials Programme, the stated aim of which was to support coordinated research into 5G technology and use cases among British telecom businesses, equipment manufacturers, and scientists. By their account, the creation of these 5G testbeds provides a crucial proving ground for technology that’s still taking shape, and whose full uses haven’t come close to being explored yet. The UK envisions a world in which 5G powers increased connectivity in rural areas, on roads, and along rail lines, as well as smart tourism and smart cities and towns, but they won’t know whether these goals are realistic until they’ve had the chance to test things out in a safe environment.
When it comes to banking and financial services functionality, “almost” doesn’t count. From your end users’ perspective, they’re either able to complete their desired action—whether that’s checking their balance, transferring funds, or setting up automatic payments—or they’re not. They’re not going to spend a lot of time looking for workarounds, they’re simply going to register their displeasure by choosing a new app or a new service provider.