RootMetrics Puts Carriers to Test at CES
To see just what kind of impact a crowd like CES can have on mobile networks, RootMetrics set out last week to test the largest commercial wireless networks and measure download and upload speeds as well as data connection failure rates on smartphones.
The upshot? Turns out some networks were so loaded that Root wasn't able to do what it originally planned to do, which was to publish four reports from Jan. 7 through Jan. 10 from the show. The transmission of its test results depended on the very networks that it was measuring, and "due to significant congestion and traffic pressures on some networks during the show," it wasn't able to provide those daily reports.
But the Bellevue, Wash.,-based mobile performance measurement company did compile the results for release yesterday, and in a snapshot, T-Mobile USA provided the fastest 3G service at CES; Verizon Wireless won the distinction of most reliable 3G; and Sprint earned the title of not only fastest 4G (provided via Clearwire's WiMAX), but also the most reliable 4G. Root said for the purpose of its report, it tested smartphones marketed as 4G devices. Its goal was to examine how consumers truly experienced data networks as they conducted typical business and entertainment with their smartphones during the show.
AT&T is absent from those awards, but a note in Root's report says the large consumer use of a specific device (i.e., the iPhone) could have a significant impact on data performance on some networks. "Regardless of these factors, our measurements accurately support how consumers truly experienced each network during CES," Root said.
Root said that overall, T-Mobile's data network proved to be the fastest 3G network, and speeds recorded on Saturday and Sunday were 2x to 3x faster than other carriers. Verizon's 3G network was second fastest, followed by AT&T and then Sprint.
In terms of reliability, Verizon performed with access failure rates at less than 10 percent each day and three out of four days recorded data access failures of less than 2 percent. AT&T's 3G network measured an overall 89 percent data failure rate during the first day, with Sprint and T-Mobile recording 45 percent and 53 percent data failures, respectively, for 3G.
Root also found that 4G networks, as promised, did perform significantly better than 3G. On average, 4G download speeds were 2x to 10x faster than 3G speeds during the conference and often higher during off-peak hours. Taking data speeds and network reliability into account, Root awarded Sprint for Fastest 4G Data Network at the event. Root said nearly all the networks became most taxed during the afternoon hours, between 1 and 5 p.m., but Sprint's WiMAX network held up well throughout each day.
In the report listing its results, Root said CES represents an extreme scenario of a consumer's mobile experience and should not be considered representative of an overall carrier's network, nor overall experience for the Las Vegas area.
Devices used during the tests were the Nexus One on AT&T's 3G network; the Evo with 3G mode enabled for Sprint's 3G; the Nexus One on T-Mobile for 3G; the Droid X on Verizon's 3G; the Evo 4G-mode enabled on Sprint's 4G; and the G2 on T-Mobile's 4G-designated HSPA+ network. The study did not include Verizon's LTE network, which does not yet have commercially available smartphones.