Mobile Device Lab Testing Becomes Scalable, Smart and Subscriber-Centric
Today’s mobile user experience is driven by a complex series of interactions between networks, devices, services and applications, all of which are tested prior to products being introduced to the market. The industry is undertaking an unprecedented number of technology changes at each of these layers:
•Networks—operators are aggressively upgrading networks from 3G to LTE, and from LTE to LTE-Advanced to meet insatiable demand for bandwidth
•Devices—manufacturers are in an arms race to deliver differentiated products capable of utilizing faster, more sophisticated networks
•Services and applications—VoLTE and RCS are being rolled out to compete with OTT applications and minimize unpredictable impacts on the network.
These developments are forcing increases in the scope of testing, when industry participants are already faced with competitive pressure to reduce time to market. Testing is also no longer confined to the pre-launch phase of a product’s lifecycle: a mobile device, for example, might require testing once it has been released in market, either due to recurring troubles being reported by subscribers, or due to software updates. Mobile operators also need to ensure that new devices do not have adverse impacts on their networks. So how do industry participants stay ahead of the constantly shifting testing curve, without sacrificing time to market, subscriber satisfaction or budget?
The answer lies in making the test plan smart, scalable and subscriber-centric:
•Scalable—A seamless, integrated test platform built to incorporate tests of new technologies or applications without major upgrades
•Smart—Diagnostics and analytics which intelligently connect end-user experience metrics with device and network KPIs to facilitate root cause analysis
•Subscriber-centric—As the underlying technologies grow in complexity, performance of a device needs to be evaluated through the prism of real-world experience: under environmental conditions and through applications that effectively represent typical subscriber scenarios.
Devices are first tested in controlled lab environments, and subsequently on the live operator network. To meet time-to-market and budget constraints, the lab should get ever closer to effective representation of the real world, to minimize the number of issues that need to be caught in the less controllable live environment.
Traditionally, testing of mobile devices in the lab occurs in conducted mode, whereby the radio frequency (RF) port of the device is opened and connected to the testbed via a cable. This method has delivered sufficiently accurate results but has always ignored the impact on performance of a critical component of the device: the antenna and RF front end. Over-the-air (OTA) testing entails running the same type of tests but with network information delivered through the antenna, with the device in a shielded chamber to enable more control over environmental factors.
Azimuth’s Intelligent Over-The-Air (iOTA) platform augments standard OTA testing by leveraging the company’s expertise in lab testing of mobile device performance. In an integrated, seamless configuration, iOTA includes:
•Replacement of standard or canned channel models with playback of network conditions recorded on drive routes across a live network, to emulate actual and familiar environments
•Automation of applications—such as voice, SMS, data and video—to mimic subscriber usage, together with the ability to conduct precisely synchronized testing of multiple devices for true benchmarking
•Analysis of device-network interactions to correlate user experience issues with underlying conditions for more efficient root cause analysis.
By enhancing prediction of performance in subscriber scenarios under real-world conditions, iOTA enables more efficient development, qualification, marketing and troubleshooting of mobile devices. Device manufacturers and component suppliers can validate or revise designs further upstream in the development process. Operators can ensure that devices perform as expected on their networks prior to launch, troubleshoot issues found once devices are in market, or identify root causes for returns.
According to operator customers, Azimuth’s iOTA test results reveal performance differences between devices that are more predictive of experience in the field, and that are not made obvious in conducted testing. Azimuth’s own independent testing—comparing the performance of a set of devices on a drive test on a live network, with the performance of the same devices on iOTA replaying the network conditions of that drive test in the lab—demonstrates that results from iOTA can be used to predict relative performance among devices, with the efficiency of a contained, controlled test environment. The charts below compare data throughput of three commercially available smartphones tested synchronously: first on a drive route across a live network; and then on the iOTA platform, playing back the precise network conditions recorded on that drive route so the devices perceive they are on the live network. Although the results cannot be expected to be identical, the data reveal that the relative performance of the devices is consistent, and therefore the iOTA platform can be used as a reasonable predictor of performance in the real world.
The last decade has seen unprecedented, disruptive developments in mobile technology, and 2014 promises to be no exception. Network infrastructure vendors are delivering complex new architectures including carrier aggregation, small cells and heterogeneous networks. In North America, operators have committed to introducing LTE-Advanced and VoLTE. Globally, device manufacturers are readying products which will work on any network, anywhere. And every day brings a new, independently developed app whose impact on the subscriber experience, on the device and on the network are inherently unpredictable. All these participants in the industry need to maintain control of their products’ performance and success in the marketplace, through test paradigms that are scalable, smart and subscriber-centric.
Amit Malhotra is Vice President of Marketing & Product Management at Azimuth Systems, Inc.