Managing QoE under disruptive conditions

Telecoms.com periodically invites expert third parties to share their views on the industry’s most pressing issues. In this piece Scott Sumner, Director of Commercial Insight at EXFO, looks at the challenges of ensuring quality of experience in the current unique circumstances.

During times of crisis—pandemics, conflict, natural disasters—service providers face challenging conditions at a time when their customers are heavily dependent on telecom services. Abnormal, dynamic traffic patterns trip off seas of alerts.

When subscribers are forced into new routines, traffic volumes spike in locations and at times that would otherwise be relatively quiet. The population turns to social media, voice and video calls to check on loved ones and establish response plans with their employers. Remote working and streaming video consumption drive high utilization in residential areas.

While users may understand that networks are facing unprecedented demand, telecom brands will be punished if they fail to deliver consistent performance.

Net promotor score (NPS)—a key metric closely tied to churn, customer lifetime value (CLV) and profitability—is 49% influenced by mobile customers’ network experience. When quality of experience (QoE) is not perceived to be ‘very good’, promotors become detractors.

Telecom brands can be quickly damaged, but not quickly repaired: when subscribers leave their provider they will not return for at least five years. Retention costs rise, and margins erode. [sources: IPSOS, and Portevo, 2019]

Rising complexity

Addressing service degradation and outages is hard when operations staff are working remotely, and deafening alarm noise impedes their ability to isolate fault origins. Maintaining peak performance under these conditions strains existing processes that are largely manual and serial in nature.

Consider a recent survey by Heavy Reading that showed operations teams typically use six service assurance tools and require 12 experts from three different domains to resolve even minor outages. Lots of emails, SWAT teams, spreadsheets… it feels like wartime conditions even under normal circumstances.

Current crisis conditions foreshadow the traffic patterns that will be introduced by highly dynamic 5G services and the new enterprise and IoT applications they will support. The GSMA notes that by 2025 there will be five times more machine ‘mobile subscribers’ than humans, and network traffic growth is projected at 53% per year over the same period. We won’t be returning to the same normal we’re used to, no matter how we manage our way out of the current pandemic.

With an increasing degree of virtualization and orchestration adding to the mix, the complexity operations teams face is beginning to exceed their resources—both tools and staff. So, the vast majority of service providers are turning to automation and machine learning approaches to prepare for a continued state of ‘abnormal’. This strategy also gives their scarce experts superpowers to reveal and resolve issues proactively, even prescriptively.

Refocusing efforts

One key lesson learned is that the focus of operations teams needs to change. Instead of alarms, they need to focus on customer-impacting events. At first glance, this may appear similar: lots of alarms should indicate a fault affecting many customers. But there is often a shaky correlation between alarms and quality of experience.

Consider a cell tower that loses power from the grid. A series of alarms will kick off. Is this a major problem? It’s hard to tell. Cells are made to be redundant; neighbors can often pick up subscribers. Maybe the cell is still working fine on a generator. Maybe it’s totally down, but at a location or time of day where there are few people even using it.

This simple example illustrates why operations teams are learning they need to prioritize action based on the actual number of customers impacted, not simply network faults. It also means they need to detect degradations. These short term ‘silent failures’ are significantly more impactful to customers trying to ‘get things done’ than the rare outage.

In a recent incident at a North American mobile operator, a significant number of customers were unable to place or receive calls. What started as a small service issue rapidly ramped into a large-scale problem—but remained undetected by the operator for nearly three hours.

This is typical to degradations that occur under high utilization conditions. The issue affected a relatively small density of customers across a large geographic region—resulting in individual experiences that were effectively invisible to monitoring systems.

Example of geographically-spread impact that would be undetected by KPIs and alarms based on population averages.

Under highly dynamic traffic conditions, customer-impacting events can suddenly appear and rapidly escalate, often as the result of overloaded core network functions (e.g. authentication servers, scalability of vIMS functions).

This requires a new way of detecting, assessing and resolving customer-impacting events quickly, and directly—not inferred from network metrics—before they escalate into large scale outages.

It’s a difficult problem for traditional big data analytics. Certainly, this is a big data problem, but the approach of storing metrics and data feeds then looking for user experience issues is compute- and time-intensive.

Real-time analytics – reveal the invisible

Machine learning methods that use stream-processing platforms like Apache’s Spark are designed to overcome these limitations by analyzing data as it arrives, in real-time. This is where innovation is centering on a new breed of customer experience analytics. The wealth of available open-source analytics and machine learning platforms are redefining what is possible, and scalable.

Operations teams are now experimenting with real-time detection tools that use machine learning to detect QoE issues, assess the impact, and diagnose the root cause within minutes. This permits operations to see which issues are most important to address, and what action to take—without sifting through alarms or consulting multiple monitoring systems.

Machine learning is allowing them to see through a new lens that derives significantly more insight from existing systems. This allows operations teams to resolve issues faster and deliver an excellent quality of experience that drives loyalty, subscriber count and margin in challenging times. When things will unlikely ever be ‘normal’ again, it’s good that there are new tools emerging that will help us remain in control.

 

Scott Sumner is the Director of Commercial Insight at EXFO, covering strategy, sales enablement and analytics. Scott has extensive experience in wireless and mobile telecom, AI and analytics, and service assurance. He has over 20 years of industry experience, including in roles as Director of Strategy and Communications at Nuvoola, VP of Marketing and Analytics at Accedian, GM of Performant Networks, Director of Program Management at MPB Communications, VP of Marketing at Minacom (Tektronix), as well as project and engineering management roles at PerkinElmer (EG&G). Scott has led numerous acquisitions and industry partnerships, and has authored numerous patents and conference papers.

Exfo uses AI to reassure 5G operators

Testing and service assurance vendor Exfo has launched some new cleverness designed to take the stress out of managing a 5G network.

In case nobody told you, 5G is a lot more complicated than any of the previous Gs, so much so that it’s just too much for mere people to get their heads around. That’s where artificial intelligence comes to the rescue, with its omniscience and ability to learn on the job. Exfo reckoned it was about time its service assurance platform made the most of AI so it has launched Nova Adaptive Service Assurance.

The cleverest bit of it seems to be Nova SensAI (possibly a play on the word ‘Sensei’), which Exfo describes as its central nervous system. As you may have guessed, it’s all about using AI and machine learning to analyse the many layers of the network and offer a good view of them. Exfo claims it will uncover network issues no other equivalent platform can, possibly even before they’ve happened.

“The combination of more users, more connections, more apps and more convoluted networks has created a perfect storm of complexity for operators,” said Philippe Morin, EXFO CEO. “By delivering only the right data at the right time, Nova A|SA is a unique intelligent automation platform to provide operators with 100% visibility into user experience and network performance. We’re talking about operations teams being able to resolve issues in minutes rather than days—or preventing them entirely.”

We’d be lying if we said we had any way of verifying those claims, but as the nature of the launch implies, this is all very complicated stuff. We do know that Exfo is up against some pretty stiff competition in the 5G service assurance space, with all its competitors also claiming to take the stress out of 5G for operators. Telecoms CTOs would seem to have their work cut out picking the best one.

Ericsson upgrades Radio System, partners with Juniper on backhaul and buys CENX

Ahead of MWC Americas Ericsson has embarked on a frenzy of announcements around its core product offering.

The headline news is a significant upgrade to the Ericsson Radio System, its signature RAN product suite that has been a major part of its apparent recovery. Specifically Ericsson is launching something called the RAN Compute portfolio, which consists of a couple of baseband processors and a couple of radio processing units designed to be positioned wherever in the network you want your processing to be done. In other words this is a mobile edge computing play.

The other big thing in new, improved ERS is some new software called Ericsson Spectrum Sharing. This is designed to help with dynamic support of both 4G and 5G on the same spectrum, so long as you’re using ERS shipped since 2015, and can be installed remotely. While some of 5G will take place on higher frequencies, the stuff currently being used by 4G has the best propagation characteristics and will therefore remain valuable. This is the kind of 5G software upgrade Ericsson has been promoting as a key feature of ERS from the start.

“The hardware and software that we are launching today continues to address the flexibility needed for the next-generation networks,” said Ericsson EVP of Networks Fredrik Jejdling. “They offer our customers an expanded and adaptable 5G platform, making it easier for them to deploy 5G.”

We had a chat with Nishant Batra, Head of Product Area Networks at Ericsson, ahead of the announcement and he stressed this is all about ramping ERS’s 5G capability. Initially the propaganda was all about it being 5G upgradable, then about being ready for the 5G launch. Now the narrative revolves around this kit being positioned for the mass deployment of 5G.

Ericsson wants the world to see a picture of growing positive momentum and trying to be the perceived leader in 5G kit is a key part of that. “The momentum has never been better and we want to keep accelerating,” said Batra.

All this RAN shininess isn’t much good without some top-notch backhaul, however, and nobody is claiming that as an Ericsson strength. 5G is set to massively increase the volume of data passing across networks so, which being sure to big-up its own Router 6000 backhaul product and microwave tech, Ericsson has announced the extension of its partnership with Juniper to augment its transport efforts, as well as a new partnership with ECI on the optical side. So much for the big Ericsson Cisco partnership eh?

“Our radio expertise and knowledge in network architecture, end-user applications and standardization work put us in an excellent position to understand the requirements 5G places on transport,” said Jejdling. “By combining our leading transport portfolio with best-in-class partners, we will boost our transport offering and create the critical building blocks of next-generation transport networks that benefit our customers.”

“Commercial 5G is expected to represent close to a quarter of all global network traffic in the next five years,” said Manoj Leelanivas, Chief Product Officer at Juniper Networks. “With both companies bringing together industry-leading network technology, Juniper and Ericsson will be able to more effectively capitalize on the immense global market opportunity in front of us and help our customers simplify their journey to fully operational 5G networks.”

In other Ericsson news it has indulged in a rare bit of M&A via the acquisition of US service assurance vendor CENX. This move is designed to augment Ericsson’s OSS and managed services offerings and CENX is all about cloud-native automation, so its technology and 185 staff should be especially helpful in the area of virtualization. They haven’t said what it cost.

“Dynamic orchestration is crucial in 5G-ready virtualized networks,” said Mats Karlsson, Head of Solution Area OSS at Ericsson. “By bringing CENX into Ericsson, we can continue to build upon the strong competitive advantage we have started as partners. I look forward to meeting and welcoming our new colleagues into Ericsson.”

“Ericsson has been a great partner and for us to take the step to fully join Ericsson gives us the best possible worldwide platform to realize CENX’s ultimate goal – autonomous networking for all,” said Ed Kennedy CEO of CENX. “Our closed-loop service assurance automation capability complements Ericsson’s existing portfolio very well.”

Lastly Ericsson has announced a new partnership with US operator Sprint to build a new virtualized core and operating system dedicated just to IoT. Network slicing will be a major feature of the 5G era and IoT has network requirements quite distinct from other usage models, so it makes sense to not just apportion a piece of the network to it, but customise all the other tech too.

“We are combining our IoT strategy with Ericsson’s expertise to build a platform primed for the most demanding applications like artificial intelligence, edge computing, robotics, autonomous vehicles and more with ultra-low-latency, the highest availability and an unmatched level of security at the chip level,” said Ivo Rook, SVP of IoT for Sprint. “This is a network built for software and it’s ready for 5G. Our IoT platform is for those companies, large and small, that are creating the immediate economy.”

“Sprint will be one of the first to market with a distributed core network and operating system built especially for IoT and powered by Ericsson’s IoT Accelerator platform,” said Asa Tamsons, Head of Business Area Technology & Emerging Business at Ericsson. “Our goal is to make it easy for Sprint and their customers to access and use connected intelligence, enabling instant and actionable insights for a better customer experience and maximum value.”

That Ericsson is making so many announcements ahead of MWC Americas would appear to be a major endorsement of the event and of the GSMA’s regional expansion of the MWC brand. The timing might also have been influenced by the staging of Huawei’s Operations Transformation Forum event and even IFA, and it’s clear there is room in the telecoms calendar for big Autumn trade fests.