It’s no secret that test quality has a direct impact on quality of service, meaning that high quality tests can and do correlate with telco operators’ ability to attract and retain customers. And yet, as telecommunications networks become more and more complex, maintaining high quality tests for things like subscriber migrations, new network rollouts, device acceptance, etc. is becoming more difficult and time consuming than ever. Obviously, testers need to find a way to maintain coverage and quality levels—even in the face of growing network complexity—but the path to doing so is not always clear.
Network quality has long been a top cause of customer churn for telcos. Yet organizations often continue to struggle with delivering adequate quality because the demand for more data has negatively impacted voice service. That demand will likely grow; according to a recent McKinsey study, consumer demand for data will increase by 40 to 80% per year, depending on customer patterns and geographic region. While data might seem more urgent, voice is still important. Telcos that wish to remain competitive are placing new emphasis on network quality testing.
Automation is often treated like a magic bullet, a cure-all for increasing demands on testing personnel who face new network quality concerns, additional devices, and other challenges every day. However, the truth is that automating any process, especially a critical one like network testing, is fraught with pitfalls. These five best practices can help ensure the success of network testing automation.
End-to-end testing—it’s all the rage right now in any number of industries, and with good reason. As global technologies become more connected and more thoroughly interwoven into the fabric of society, more thorough and efficient testing will become not just a luxury, but a necessity. As such, it should come as no surprise that a number of testing automation providers have cropped up in the past few years, covering everything from software security to telecoms. This, too, makes sense on the surface: how different could testing solutions for different industries really be? If you can automate verification for a new social media platform, why can’t you do the same for VoIP verification?
Let’s say you’re working through a whole slate of different test scenarios to verify service on a new network that you’re rolling out. One of your first tasks is tackling VoIP (voice over IP) tests, which, as it turns out, present some very particular challenges. Because jitter and latency in voice conversations can quickly render a call frustrating and incomprehensible, your tests have to seek out extremely granular data about packet loss and packet delay for a number of different use cases. In order to do so effectively, testers need a wealth of specialized knowledge.
In an ideal world, service verification for voice, data, and mobile broadband usage would probably look a lot different than it does right now. Test cycles would be perfectly matched to the timelines for updates, testers would be able to complete tests for the entire range of use cases with time to spare, and any bugs uncovered could be addressed before new updates were rolled out. Unfortunately, that’s not really the world we live in. Instead, we’re stuck with update cycles that are often too short for thorough use case testing, and service verification begins to feel like an unwanted albatross around the neck of any given telco operator.
Network demands have grown increasingly complex in recent years. For example, people's reliance on smartphones for everything from navigation to mobile banking means that networks must be more robust and secure than ever. Meanwhile, the proliferation of IoT-enabled devices has introduced new protocols and device configurations.
According to a recent McKinsey study, network quality concerns comprise some of the most important factors impacting a given customer’s choice of mobile carrier. While pricing is still the most important one on average, survey respondents were also quick to list national and local coverage, network speed, and quality of 4G as critical deciding factors in carrier selection. In spite of the growing importance of network quality, however, McKinsey also found that the average quality of service for voice has decreased across Europe in recent years.
A few years ago, a tester working on a typical telco project could run through about 10 use cases per day. Now, that number is closer to 8 use cases per day. This trend might be worrying from the outside, but if you’re a test engineer within the world of telecommunications it really shouldn’t be shocking. After all, as the complexity of global networks skyrockets, it stands to reason that verifying service for any particular node or function would become incrementally more complex as a result. The question is: what can network operators do about it? How can you maintain standards and achieve a positive testing ROI in these increasingly difficult environments?