The chart below is, I believe, one of the more important ones ever produced in the history of packet networking. It was generated by my colleagues at Predictable Network Solutions Ltd, based on measurements taken from Kent Public Service Network (KPSN), and reproduced with their kind permission. It is the first publishable direct comparison of standard network management utilisation data against a strong user quality of experience (QoE) proxy. This data is important because it contradicts a common belief about networking.
The widespread assumption is that as long as you have enough bandwidth, any remaining quality issues can be readily addressed by allowing some ‘slack’ excess capacity. This is called ‘over-provisioning’. Its presumed effectiveness is the bedrock of investment plans in 4G and fibre. The problem is that over-provisioning is both ineffective and inefficient.
This is invisible to network operators, because their metrics fail to offer the necessary QoE information. This in turn costs the telecoms industry an unknown (but very large) sum in wasted capital expenditure and churn. Our engagements with other communications service providers (CSPs) have shown this to be a common phenomenon. This one chart calls into question many of the basic assumptions on user value and return on capital of network planners and investors.
KPSN is one of the best-run networks in the UK, and delivers broadband services to schools, hospitals, council offices, etc.; it has around 250,000 individual users in Kent spread over 1,100 sites. KPSN costs (in our estimation) about 60%-80% less to run compared to an equivalent managed service from a major systems integrator. (How they achieve this feat will be a subject of a future article.)
As part of their expansion and growth plan, they asked my colleagues to predict the QoE and cost effect of adding further users and uses (e.g. voice) to the network. A key step was to establish a baseline for the current QoE on offer, and measure what performance hazards were armed, or had already matured into application failures.
The chart below compares two sets of measures for one 100mbps fibre link, in one of the collector arcs of the network, taken over a five day period.
The green line is the standard 5 minute average link utilisation data that every packet switch offers by default. The red crosses are a robust synthetic metric that represents the risk of failure a national call phone call, based on a ‘performance budget’ sub-divided and attributed to this link. (The measurement techniques required are described here.)
As you can see, there is a correlation between when the network is busy, and when QoE is at risk. However, there are two key things of interest in this data, highlighted below.
The first is that there are big spikes of usage (in the evenings), but the QoE of the user application is not at risk. That means any network planning rules that use that data would force inappropriate upgrades that just waste money for no user gain.
The other is that there are potential QoE failures at even extremely low average loads (under 0.01%, i.e. over-provisioned by a factor of 10,000). Hence over-provisioning is not an appropriate way of solving QoE issues that are due to poor scheduling choices. Operators are either throwing capacity and capital at these problems without resolving them; or users are having QoE failures that are not visible to operators, and are churning.
The telecoms industry is like the proverbial drunk searching for its lost customer experience keys under the street lamp. It’s easiest to look at the single point average bandwidth data the network gives you for free, not the strong QoE proxy data that requires multi-point measures.
The bottom line is that average bandwidth utilisation is a poor proxy for user application QoE. The industry is using network-centric metrics that fail to reflect to the customer experience, and are building and operating networks based on assumptions that are only weakly founded. This wastes capital, and drives unnecessary churn.
“If you do not change direction, you may end up where you are heading.” ―Lao Tzu
For the latest fresh thinking on telecommunications, please sign up for the free Geddes newsletter.