As AT&T, Verizon and T-Mobile took shots at one another in press releases and in the pages of national newspapers, a common name kept popping up: RootMetrics. AT&T noted that RootMetrics awarded it best overall call, text and data performance in several markets when it claimed the title of “nation’s most reliable” network in a press release that drew response from Verizon. “Reliable” looks good on a network even if it can leave consumers scratching their heads.
But as RootMetrics CEO Bill Moore pointed out, the rigorous tests his company runs on the nation’s wireless networks are first and foremost in the public interest.
“We’re out to represent the consumer. It’s all about consumer advocacy,” Moore said. “Everything we look at is from that consumer perspective.”
That means that all the data RootMetrics generates from its testing is provided for free to consumers. And it’s a lot of data, coming in the form of reports (for each market and in summation) and coverage maps. It’s all available online or via free mobile applications for iOS and Android.
RootMetrics just completed the first half of its year, covering 125 of the top markets in the U.S., a geographic segment home to 177 million wireless consumers. RootMetrics takes an off-the-shelf Android device—whichever benchmarks as the best for each network, that in most cases was a Samsung Galaxy device—and tests the voice, text and data service of each network in all conceivable locations, times and conditions.
“When we go out and do our direct testing, we do it on a level playing field for all the carriers,” Moore said.
RootMetrics time-syncs the tests as well to make sure the four big carriers are all being tested simultaneously. Running the same tests in the same place at the same time on the same device on all four carriers is the big distinction that sest RootMetrics apart from other network testers, Moore said. He added that running the call, text and data test on the same device is important for RootMetrics because that’s exactly how consumers use their devices.
When the testing is finished, the reports are made and the maps are updated and RootMetrics begins the next six-month cycle of testing, hitting the same 125 markets again. But first, RootMetrics hands out awards for call, text, data and best overall in each market. Not too surprisingly, AT&T and Verizon trounced Sprint and T-Mobile in RootMetrics newest reports. Still, Moore is careful to reiterate that RootMetrics has no network bias.
“All of our methodology is transparent in our reports. We are not only a consumer advocate, if you will, we are fiercely independent,” Moore said. “No one tells us how to do our methodology. We get it validated and we have PhD statisticians on staff on ensure we do this with high statistical integrity. And we’re very transparent about what we do.”
Moore added that RootMetrics is much more comprehensive than the “woefully inadequate” tests performed by its competitors. He was equally skeptical about the lofty claims of reliability some carriers throw around.
“There’s a lot of claims out there right now about what reliability is,” Moore said. “And we think there’s a small amount of definition about what [reliability] means or no definition at all.”
Moore said RootMetrics aims to shore up the ambiguity by soon publishing a set of standards defining network reliability. RootMetrics’ definition of reliability will be based on its own findings as well as consumer input.
“We’re going to make very clear what we think is not very clear as a standard in a consumer’s mind,” Moore said.