Measuring Broadband Quality

What defines good broadband, and, from a public policy standpoint, how much is it worth? Broadband customers know good – or bad – broadband when they see it, and, as even a quick glance at online ratings sites will confirm, they are not shy about expressing their opinions. But translating customer satisfaction levels into public policy is tricky.

Two new studies commissioned by the Fiber Broadband Association (FBA) help put the measurement of broadband quality on a sounder footing. The FBA, a trade association that represents more than 250 vendors and service providers in the fiber-to-the-home industry, undertook the studies in response to the FCC’s proposal for Rural Digital Opportunity Fund (RDOF) auctions.

In the rural broadband auction conducted most recently (CAF II), the winning bidder in each area was the provider that requested the lowest level of subsidy. (That’s what makes it a “reverse” auction.) To incentivize high-performance networks, however, bidders received extra credit for proposing infrastructure capable of high speeds and low latency. The FCC’s proposal for the new, much larger RDOF auctions included the same type of weighting scheme, although the tiers and credits were adjusted somewhat. The FCC solicited comments about whether the tiers and weights were appropriate.

Not surprisingly, the satellite provider Viasat, which won large awards under CAF II, wanted to see smaller credits for high-performance networks, and several wireline providers, along with the FBA, argued for larger credits. The FBA was apparently the only organization to back up its recommendations with an economic analysis.

FCC Undervalues Speed and Latency

The FBA asked the business consulting firm Cartesian to study whether the weighting of high-performance credits in the CAF II auction achieved the FCC’s goals for the auction, and whether the weights for the RDOF auctions would do so. The FCC’s stated goal is to make the most efficient use of its limited budget by maximizing auction participation to drive down prices and reflect the value of higher-tier and lower-latency services.

Cartesian found that, despite rapidly growing demand for high speeds, the weights the FCC used in its CAF II auction did not encourage meaningful participation by providers offering gigabit services. Most winning bidders in the gigabit tiers were electric co-ops, which had a cost advantage because they were already building fiber networks to support smart-grid services. More than 80 percent of the locations in the auction had no bids for gigabit services, and few had more than one.

Had more gigabit providers bid in the auction, Cartesian said, the increased competition would have driven prices down, allowing the CAF to serve more locations with better broadband without exceeding its budget. Furthermore, winning providers might ultimately have been able to expand services beyond the auctioned locations.

To determine whether the FCC set credits for high-performance broadband at appropriate levels for CAF II, Cartesian estimated the value of broadband in seven use cases: telecommuting, remote health and learning, e-commerce, streaming video, cloud storage and computing, gaming and social media, and two-way video streaming. It calculated an average value of $812.50 per person per year for gigabit fiber service, compared with $75 per person per year for minimum satellite service (other types of infrastructures yielded values within that range). However, the FCC weights disproportionately rewarded mediocre infrastructure, including baseline satellite service, and failed to take into consideration the superior performance of infrastructure that could provide gigabit and higher speeds.

Going forward into the RDOF auctions, Cartesian recommended a recalibrated weighting system that would increase the penalties for baseline and minimum speeds. Such a revised weighting system, Cartesian concluded, would encourage more gigabit providers to bid. The resulting increased competition would reduce overall bids and allocate funds to maximize consumer value.

RVA measured speed, latency and reliability, and asked consumers how satisfied they were with each performance attribute.

 

The Broadband Experience Index

Are speed and latency the only performance measures the FCC should use? To find out, the FBA turned to market research firm RVA LLC.

RVA has been conducting consumer broadband surveys for many years, and it recently developed a method for surveyers to conduct speed and latency tests without respondents having to initiate the tests. This greatly improves the accuracy of the information gathered. Over the years, RVA has consistently found that consumers value reliability even more than speed and latency.

In the current survey, in addition to measuring speed, latency and reliability, RVA asked consumers how satisfied they were with each performance attribute. Finally, it calculated for each broadband modality a “net promoter score,” a widely used indicator of consumer satisfaction based on the difference between the percentage of respondents who would recommend a product and the percentage who would not. (For example, if half of respondents say they are very likely to recommend a product to a friend and half are unlikely to recommend it, the net promoter score is zero.)

To construct an overall broadband experience index for each modality, RVA averaged its own speed and latency measures with the FCC’s speed and latency measures, consumers’ reports of reliability, and the net promoter score for each modality. Fiber to the home had the highest score of all modalities, with a 98 percent rating. The low performance and consumer dissatisfaction with satellite service yielded a 1 percent broadband experience rating. Cable, wireless and DSL service scored 65 percent, 38 percent and 38 percent ratings respectively.

These findings supported the FBA’s contention that “higher-tier services provide better performance and greater satisfaction to consumers,” adding further evidence that the FCC should increase the weight it gives to auction bids for high-performance networks.