As of a few years ago, more than 90% of software testers reported that they were automating between 50 and 100% of their tests. Of the survey respondents who had automated, about a quarter saw ROI immediately, another quarter within 6 months, and another quarter within a year. Fewer than 10% failed to reach ROI. Naturally, the telecom domain is its own beast, and in all likelihood the numbers for automation adoption would look a little bit less robust—but an examination of similar trends adjacent to the telco industry can still be telling for network operators and testers.
One trend in particular that telco testers might want to take notice of is the introduction of robots into many testing frameworks. Or, more specifically, the use of Robot Framework as a building block for improved testing workflows. One blogger described Robot Framework as the unsung hero of test automation, and given the way that these robots are able to power keyword driven test suites, it’s hard to argue with that claim. Anyone considering automating their network migration tests or their interworking verification would do well to take note.
What Is Robot Framework?
Okay, let’s back up a step. What exactly is Robot Framework, and why should testers care about it? Basically, it’s “a generic open source automation framework for acceptance testing, acceptance test driven development (ATDD), and robotic process automation (RPA).” At its heart, it sets out to solve a very specific problem: code reuse. In earlier incarnations of test automation, most testers were stuck with linear frameworks that involved huge blocks of single-use code. Essentially, you would script up all of the actions that needed to happen to the system under test, your automation framework would run through the entire block of code in order, and then you’d be stuck trying to decipher the results. There were no control structures in place to empower users to create generic code, which meant that every new round of tests required a new round of laborious scripting of use cases.
Naturally, this is an issue that has historically plagued software testers more than telecom network testers, but with automation becoming an increasingly common fact of life, the same concerns begin to apply. If you can’t reuse your code, you’re essentially reinventing the wheel for every round of service verification. Within the Robot Framework, this issue essentially disappears. By creating a library full of keywords (each of which contains a reusable function), you can abstract out all of the actions that manual testers would traditionally have had to do themselves, and early automation users would have had to rescript on a granular level over and over again. Thus, if a tester wants to verify that a call can be placed on an updated network between a mobile subscriber and a landline user, she simply uses the keywords for each of the network elements and the actions they need to take. From there, the framework controls the relevant device to perform the actions and spits out reporting generated with those same keywords, meaning the results are fairly easy to read even for non-technical users.
How Do Robots Simplify Test Automation?
Considering how complex telco testing has become in the past several years, it seems safe to say that anything that can truly simplify (or, better yet, speed up) service verification is going to have a strong demand. So how exactly do robots simplify testing? The key is finding the right level of abstraction. On a granular level, keyword-based functions are actually a collection of smaller actions that have been abstracted into a single command. For instance, if you want to test that a user can log in to an app on your network, that would ostensibly require the device user to unlock the phone, click on the app, click the form fields on the log-in page of the app and type out the appropriate information, then click the log-in button. If each of these steps had to be specified individually for each test case, things would be slow and cumbersome—but all of these actions are already contained within the relevant keyword for logging in. This means that testing this functionality on a host of different end user devices (to ensure that there’s no acceptance issues) would just be a matter of repeating the keyword based action with the relevant device-based keywords and reading the results.
As you can imagine, this scales more effectively than many alternatives for two reasons: it maximizes code reusability, and it lowers the technical barrier to entry so that everyone working for a given network operator can perform and understand basic tests. Because this blurring of boundaries can help telco operators to cover more areas of service verification more quickly in an era where high test coverage is becoming increasingly difficult to achieve, we expect that Robot Framework will become increasingly entrenched in testing flows for telecoms—with the end result being improved testing ROI and boosted network quality.
Implications for Future Testers
Okay, so the answer to our initial question ("Why will robots shape the future of telco testing?") turns out to be pretty straightforward: because they’ll help power the keyword based testing flows that can keep network tests manageable in terms of time and resources. But what implications does this have for testers going forward?
Basically, two things: First, that if you’re considering automation or you’re refining your existing automation workflows, you should consider giving stronger consideration to solutions that run on Robot Framework and offer keyword based tests. Second, that as testing becomes faster and more democratized, it will be incumbent upon testers to help put not just technical but operational frameworks in place to ensure success. This means that you’ll need to help define who’s in charge of what, when; you’ll have to create and maintain standards for test scripts; and you’ll have to maintain discipline about storing test results appropriately and delegating bug fixes. If you can manage this, you can turn automation into a serious competitive advantage.