When wireless carriers look at the results of third-party network testing, they might be inclined to hear what they want to hear. Such is the case with PC World's latest round of 3G Wireless Performance tests, which the magazine conducted in partnership Novarum, an independent testing firm.
AT&T made out big in these latest tests, putting up download speeds that doubled what was seen out of the carrier's network last spring. If you're Ralph de la Vega, CEO of Consumer Mobility at AT&T, you're telling your PR to send the report to every journalist and blogger they can find. If you're Verizon Wireless, and your speeds actually decreased since PC World's pervious report last spring, you're trying not to put too much credence into the results.
Tom Pica, spokesman for Verizon corporate communications, was dismissive of the results. He said that Verizon's own testing tells the story of a network that is larger and more dependable than any other network in America.
"I don't get this stuff, I mean why 13 cities? Who picked these cities? … Those are nice places, but we test everywhere," Pica said. "We drive a million miles a year, testing our networks and our competitors’ networks, and that's the basis for our claim of 'Nation’s largest and most reliable 3G network.'"
Pica said that Verizon takes into account a range of testing results that covers a lot more ground than 13 cities.
"I don't know the methodology here... There are a lot of companies out there claiming to provide people with results. We look at them. We look at our results. We look at a lot of third-party testing results, and in the end, we have the largest and most reliable 3G network. That's the story."
Ken Bida, co-founder and CTO for Novarum, said there's a total lack of third-party data by which to gauge the reality of the networks. He said that the PC World data is an attempt to take an accurate look at how the networks are performing in a time when data demand is experiencing exponential growth.
Bida said that this most recent batch of testing, which was done in December 2009 and January 2010, duplicated the parameters set in its testing last spring. He said the doubling of AT&T's average speeds, from 818 kbps to 1.4 mbps, is evidence that suggests AT&T's most recent improvements really are making a difference.
"I think the pieces we clearly noticed were that AT&T, in almost every city, was markedly faster. Clearly, AT&T has done something since [our last testing] to make a difference," Bida said.
Verizon, on the other hand, experienced a drop in speed in almost every market tested. Bida speculates that Verizon Wireless might be having a hard time keeping up with the data demand it's seeing from a host of new smartphone customers.
"My speculation is that, yes, they're seeing the effect of more smartphones on the network. They only had the Droid for about two months when we measured it. So we were probably measuring about 1 million handsets? The piece that was compelling to me was in how widespread the lower scores were for Verizon," Bida said.
All signs point to AT&T's HSPA 7.2 software upgrade as a primary variable between the tests a year ago and this most recent batch. Bida also said it's possible that AT&T's extensive network of Wi-Fi hot spots could be helping with off-load.
To be fair, Verizon is actually on track to be the first U.S. carrier to deploy LTE on any scale. But Bida says that may not mean the kinds of blazing speeds some people might expect.
He notes that Verizon's projected LTE footprint of 35 cities by the end of 2010 is almost identical to that of Sprint and Clearwire’s current WiMAX offerings. Bida said that Novarum tested Sprint's WiMAX service alongside other 3G networks in its 13 test markets and found that download speeds for Sprint came in at about 2.2 mbps, whereas uplink speeds were actually slower than AT&T's 3G.
"No one really knows what 4G is going to mean," Bida said, adding that the blanketing of a variety of technologies as 4G is somewhat misleading.
Bida said that because there's so much happening in mobile right now, Novarum intends to repeat its tests on a more frequent basis. He said he's particularly interested in seeing what effect, if any, Apple's iPad has on AT&T's network once the device ships in March.