End-to-end testing: for many telco operators it’s the holy grail of service verification, but it can also be a slow, laborious process that adversely impacts time to market. Even if you’ve managed to automate your relevant equipment and collect success and failure data from the relevant end-points, you might still find yourself in a position where hard-to-read data and hard-to-program use cases stop your end-to-end tests from running as quickly as you would like. When this happens, you’re in the uncomfortable position of either sacrificing high levels of test coverage by cutting the test off early, or delaying your network migration or device rollout to accommodate slow testing.
One of the top goals that every telecom operator aspires to is consistent service, and a big part of that consistency is tied to how well you can coordinate with other networks to offer high quality roaming service for your customers. Perhaps more so than in the past, users don’t want to comb through a lot of fine print about where their in-network coverage begins and ends—they simply want to be able to use Gmail while they’re out and about in the world without experiencing any glitches or service anomalies.
It’s no secret that test quality has a direct impact on quality of service, meaning that high quality tests can and do correlate with telco operators’ ability to attract and retain customers. And yet, as telecommunications networks become more and more complex, maintaining high quality tests for things like subscriber migrations, new network rollouts, device acceptance, etc. is becoming more difficult and time consuming than ever. Obviously, testers need to find a way to maintain coverage and quality levels—even in the face of growing network complexity—but the path to doing so is not always clear.
The modern cycle of updates for telecom networks continues to speed up, and telco operators need to do the same in order to keep pace with the market. For some of you, this may be leading you to consider test automation for verifying service on your network. Sure, there’s plenty of content out there about automated testing, and some of if even pertains to the specific challenges that your company has—but it’s still more than a little bit daunting. You know you need a framework that can automatically test, for instance, audio quality for VoLTE service across a suite of modern and legacy devices, but how do you get started?
Network quality has long been a top cause of customer churn for telcos. Yet organizations often continue to struggle with delivering adequate quality because the demand for more data has negatively impacted voice service. That demand will likely grow; according to a recent McKinsey study, consumer demand for data will increase by 40 to 80% per year, depending on customer patterns and geographic region. While data might seem more urgent, voice is still important. Telcos that wish to remain competitive are placing new emphasis on network quality testing.
Let’s say you’re working through a whole slate of different test scenarios to verify service on a new network that you’re rolling out. One of your first tasks is tackling VoIP (voice over IP) tests, which, as it turns out, present some very particular challenges. Because jitter and latency in voice conversations can quickly render a call frustrating and incomprehensible, your tests have to seek out extremely granular data about packet loss and packet delay for a number of different use cases. In order to do so effectively, testers need a wealth of specialized knowledge.
Okay, you’ve decided to take the plunge. You know that in order to keep pace with all of the new devices, network protocols, and use cases emerging every day in the telecom domain, you need to do something about your testing framework. Your internal engineers can’t manually test all the required use cases anymore, and service verification is only going to get more complex as technologies like 5G enter the scene. You’ve even done your research via Google and your professional network to see who the worthwhile vendors might be for a telecom testing solution. Now you get to the hard part: how do you choose between them?
Let’s say you’re a telco service provider: after careful deliberation, you decide to migrate your network in order to improve bandwidth for your growing customer base. After some time, the hard part appears to be over—you developed a plan that involved key stakeholders, you sketched out the scope of the migration, and you updated all of your switches and other equipment as needed. Now, it’s just a matter of verifying that you’re still providing all of the services you think you’re providing.
A few years ago, a tester working on a typical telco project could run through about 10 use cases per day. Now, that number is closer to 8 use cases per day. This trend might be worrying from the outside, but if you’re a test engineer within the world of telecommunications it really shouldn’t be shocking. After all, as the complexity of global networks skyrockets, it stands to reason that verifying service for any particular node or function would become incrementally more complex as a result. The question is: what can network operators do about it? How can you maintain standards and achieve a positive testing ROI in these increasingly difficult environments?