It’s high time for the operators to focus on the customer experience metrics that matter rather than those that are just easy to measure
The telecom market in India is brutally competitive. We have seen high spectrum acquisition costs, and tariff rates causing bitter rivalry among the operators in the past. Well, the rivalry has never gone away, and rather became an intense one as the market continued to evolve further with the introduction of new telecom services, making customer experience the new battleground for the operators in the telecom sector.
The most important factor in driving brand perception and delivering a good customer experience for a telecom operator is its network experience, which is commonly referred as network speed.
Today, it is very important for the network operators to decide what use cases (e.g. video, gaming, TV, IoT, and so on) do they want their network to perform the best and assess if they are providing the right experience for that use case. Given that today more than 60% of a mobile operator’s traffic is video, this decision becomes more important than ever.
A detailed analysis on the state of video experience in India done by MOZARK shows that network speed beyond a point does not make any difference to the actual experience. The analysis is done on the basis of the performance of 6 OTTs –Youtube, Netflix, Hotstar, SonyLIV, Voot, Zee5 on three mobile operators Jio, Airtel, and Vodafone Idea.
Here are the key highlights :
Focusing on network speed alone to improve video streaming experience is not sufficient anymore:
How quickly a video loads once the user clicks on a thumbnail and whether it plays seamlessly without buffering are the most important aspects of video streaming experience. Research shows that Play Start Time (PST) below 4 seconds is what users prefer and MOZARK’s analysis shows that about 14 Mbps is what networks need to deliver this level of PST. (See Figure 1)
The rate of deterioration in PST KPI as network speeds fall below 5 Mbps was quite steep. However, once the network speeds go beyond 5 Mbps the incremental improvement in PST was quite flat. Between 5 and 30 Mbps, the PST times increase only by around ~2s which while does have a material impact on user experience is not as steep as increases of 4s seen when network speeds dropped below 5Mbps.
Maintaining a video rebuffering rate of below 4% is critical for the operators:
Research on the customer perception of rebuffering shows that when rebuffering rate increases beyond 4%, it starts to have a material impact on the user’s perception. When network speeds go beyond 14 Mbps the rebuffering rate too drops below 4%. Thus, delivering a video rebuffering rate of below 4% is critical. (See Figure 2)
Operators must focus on the Minimum Operating Threshold (MOT) for delivering the right user experience:
MOT is a function of video optimization performed at the Core and transport architecture. The study has found variations across operators in terms of the MOT i.e. the minimum speed necessary to deliver PST/Re-buffering rate necessary for quality video experience for end user. Jio, for example was able to deliver a better PST and Video Rebuffering rate at the same network speeds as compared to the other operators, especially when the speeds were low. Airtel was only marginally better than Vodafone Idea. This goes to show that investments must be made beyond the radio network, on building a strong backhaul, content delivery core and better peering partnerships with OTTs. (See Figure 3 & 4)
Collaboration between operators and apps is critical for the end user experience:
The study found significant variances between apps in terms of the PST delivered for a given network speed. Youtube and Hotstar for example, perform extremely well even under poor network conditions. Netflix deteriorates in PST performance under poor network conditions but as soon as speeds get to 10Mbps it dramatically improved in performance to match that of Youtube. Similar trends were observed for rebuffering rates too. (See Figure 5 & 6)
The data thus clearly shows the need for investments in a robust content delivery network, streamlined application architecture, and putting in place thorough testing and debugging mechanism within their organization around application experience.
A long way to go before India can deliver the MOT network speeds for ideal video experience:
Despite the improvements, we still lack the ideal network performance necessary to deliver seamless video experience across the country. The number of samples that were above 14Mbps hardly touched 20% across the country. Operators need to continue investing in expanding their site footprint, spectrum, fiber to deliver a consistent 14Mbps network speed so that Indian users can enjoy an ideal video streaming experience. (See Figure 7)
The analysis of more than 6-months network performance data of the country’s top three operators shows that the network performance clearly saw a dip during the lockdown period due to increased traffic and operational hindrances on the field. However, the performance showed an improvement trend and is very close to being back to pre-COVID-19 levels.
Preparing for the future
The MOT may vary for each use case. Network planners need to think about what use cases they want their networks to support and start designing their networks to deliver the MOT required for that use case. This in fact will become a key design factor with 5G as every use case will have a designated SLA.
The study done by MOZARK also highlights the fact that the time has come for network operators and regulators to focus on the metrics that matter rather than those that are just easy to measure. The quality of service measurements also need to become more sophisticated to not just be focused on quantity of data points of metrics that are easy to measure (e.g. network speed) but focus on measuring the end user experience.
Image & Data Source : MOZARK