According to a recent GSMA study, the IoT market will be worth $1.1 trillion and include about 25 billion IoT connections by 2025. The majority of those connections will be in the industrial and vertical industry segments (13.8 billion connections) and the smart home market (11.4 billion).
The most obvious benefits of automation for any industry include increased efficiency and decreased reliance on human employees. But for telcos, automation, and particularly automated testing, offers multiple other sources of ROI, from reduced time to market, to better implementation of the Continuous Delivery model.
Automation is often treated like a magic bullet, a cure-all for increasing demands on testing personnel who face new network quality concerns, additional devices, and other challenges every day. However, the truth is that automating any process, especially a critical one like network testing, is fraught with pitfalls. These five best practices can help ensure the success of network testing automation.
Let’s say you’re a telco operator pushing out a change to your billing platform. For many in the business world, the hope for a project like this is that the team behind it has a certain level of agility, meaning that they’re a cross-functional group that’s empowered to solve problems in a flexible manner within the company’s larger mission. Unfortunately, agility usually isn’t what we find in cases like these. Instead, we find “waterfall” projects where teams are constantly waiting for approval, wading through red tape, and carrying out pre-agreed plans even as potential challenges and hurdles come to light.
End-to-end testing—it’s all the rage right now in any number of industries, and with good reason. As global technologies become more connected and more thoroughly interwoven into the fabric of society, more thorough and efficient testing will become not just a luxury, but a necessity. As such, it should come as no surprise that a number of testing automation providers have cropped up in the past few years, covering everything from software security to telecoms. This, too, makes sense on the surface: how different could testing solutions for different industries really be? If you can automate verification for a new social media platform, why can’t you do the same for VoIP verification?
Let’s say you’re working through a whole slate of different test scenarios to verify service on a new network that you’re rolling out. One of your first tasks is tackling VoIP (voice over IP) tests, which, as it turns out, present some very particular challenges. Because jitter and latency in voice conversations can quickly render a call frustrating and incomprehensible, your tests have to seek out extremely granular data about packet loss and packet delay for a number of different use cases. In order to do so effectively, testers need a wealth of specialized knowledge.
Okay, you’ve decided to take the plunge. You know that in order to keep pace with all of the new devices, network protocols, and use cases emerging every day in the telecom domain, you need to do something about your testing framework. Your internal engineers can’t manually test all the required use cases anymore, and service verification is only going to get more complex as technologies like 5G enter the scene. You’ve even done your research via Google and your professional network to see who the worthwhile vendors might be for a telecom testing solution. Now you get to the hard part: how do you choose between them?
Smart cities. Augmented reality. Net neutrality. The Internet of Things. These are just some of the buzzwords in telecommunications right now. They all indicate a technology-driven shift in the industry, one that has fundamentally changed the successful telecom business model. As telecom companies seek to cut costs and keep customers, they must also take advantage of emerging technologies to power innovation.
In 1877, Alexander Graham Bell demonstrated the possibility of making long distance telephone calls by calling the offices of The Boston Globe. As you can imagine, the press had a field day—and it’s easy to understand why. From the perspective of technological progress, transmitting voice communication over a trunk line successfully was a feat that would have been scarcely imaginable even a few decades before, and the research and experimentation that led to it would have been extremely complex. On the other hand, service verification would have been a breeze. Since the ceiling for voice quality back then was quite low, it was still a simple matter of placing a call and hoping that it went through successfully. If so: service verified.
For many telco operators, testing can seem like an onerous requirement. It’s often costly and time consuming, and as telecom networks grow more complex and customer use cases and devices become increasingly fragmented, verifying service with any level of confidence is harder than ever. Because of this high degree of complexity, testers need to achieve higher test coverage than ever before in order to maintain network quality—resulting in the relatively widespread adoption of end-to-end testing among those in the industry. Rather than testing voice protocols and Wi-Fi connectivity from a handful of user devices, testers are walking through entire systems and subsystems in the ways that users are likely to do.
In an ideal world, service verification for voice, data, and mobile broadband usage would probably look a lot different than it does right now. Test cycles would be perfectly matched to the timelines for updates, testers would be able to complete tests for the entire range of use cases with time to spare, and any bugs uncovered could be addressed before new updates were rolled out. Unfortunately, that’s not really the world we live in. Instead, we’re stuck with update cycles that are often too short for thorough use case testing, and service verification begins to feel like an unwanted albatross around the neck of any given telco operator.
Okay, let’s say you're one of the major telco operators in your geographic area, and in order to increase your competitiveness you’re hoping to be the first one to roll out a 5G network for mobile voice and data. You’ve spent months laying the groundwork and taking pains to get your equipment and protocols in line with the new standards, and you’ve done your market research to determine the level of demand among local users (including adoption of 5G-enabled devices, etc.). It’s “all steam ahead,” and the only question is how quickly you’ll be able to get your product to market.
Network demands have grown increasingly complex in recent years. For example, people's reliance on smartphones for everything from navigation to mobile banking means that networks must be more robust and secure than ever. Meanwhile, the proliferation of IoT-enabled devices has introduced new protocols and device configurations.
Let’s say you’re a telco service provider: after careful deliberation, you decide to migrate your network in order to improve bandwidth for your growing customer base. After some time, the hard part appears to be over—you developed a plan that involved key stakeholders, you sketched out the scope of the migration, and you updated all of your switches and other equipment as needed. Now, it’s just a matter of verifying that you’re still providing all of the services you think you’re providing.
In June of last year, The 3rd Generation Partnership Project set the official standards for standalone 5G, effectively paving the way for the era of true 5G functionality. It might be a little bit of an exaggeration to say that we’re now experiencing a race to create usable 5G networks and devices among the wireless carriers and device manufacturers of the world (Apple, for instance, has been forthright about its decision to wait until 2020 to roll out its first 5G-enabled smartphone), but the floodgates are certainly beginning to open—and carriers like Verizon and AT&T are already performing a limited rollout of 5G home and mobile networks. In Europe, a leading operator in the 5G space has already announced its support for the OPPO Reno 5G.
According to a recent McKinsey study, network quality concerns comprise some of the most important factors impacting a given customer’s choice of mobile carrier. While pricing is still the most important one on average, survey respondents were also quick to list national and local coverage, network speed, and quality of 4G as critical deciding factors in carrier selection. In spite of the growing importance of network quality, however, McKinsey also found that the average quality of service for voice has decreased across Europe in recent years.
A few years ago, a tester working on a typical telco project could run through about 10 use cases per day. Now, that number is closer to 8 use cases per day. This trend might be worrying from the outside, but if you’re a test engineer within the world of telecommunications it really shouldn’t be shocking. After all, as the complexity of global networks skyrockets, it stands to reason that verifying service for any particular node or function would become incrementally more complex as a result. The question is: what can network operators do about it? How can you maintain standards and achieve a positive testing ROI in these increasingly difficult environments?
Today, SEGRON announced that it has raised 3M euros in a series A funding round from Credo and OTB. To any of you who have made use of SEGRON’s ATF to automate testing for your networks, devices, or services, we hope this won’t come as too much of a shock. Following a period in which we completed several successful commercialization trials in Germany, Switzerland, Austria, and elsewhere, we demonstrated that our product was truly the most innovative and comprehensive on the market.
In 2012, an OpenSignal study found that there were about 4,000 different Android device models on the market. Within a couple of years, that number had risen to 12,000, and it’s likely only gone up since then. As a device tester, you already know that there’s considerable diversity among your customers, and that their needs are going to vary on a case-by-case basis—but who knew there was so much diversity just in the devices themselves?