Let's Make it a Good (Mobile and Web) Experience

By Jeff Cotrupe, Program Director, OSS/BSS Global Competitive Strategies, Stratecast, a division of Frost & Sullivan

For decades communications service providers (CSPs)/operators have had the benefit of service assurance solutions providing inside-out, engineering-driven views of how their networks are performing. The market now features an abundance of resources that provide inside-out engineering views (usually with probes placed in network nodes) telling operators how their networks are performing through metrics such as uptime, bit error rate (BER), packet loss, mean time to repair (MTTR) and many more. Service assurance solutions are extremely important to operators and assist in resolving performance-impacting faults in CSP networks. Website owners, too, have had at their disposal site load testing and traffic analysis to gauge the performance of their web properties.

Yet knowing that a network or website is performing within acceptable parameters, 'all lights green' in the network operations center (NOC) or on a performance dashboard, does not completely address the performance issues end users experience. It omits users, who—if they care about performance metrics at all—would be more interested in things like:

• Dropped calls and data sessions;
• Sites that load properly on some browsers but not others, or do not load at all;
• Apps and messages that download, say, in five seconds in Caracas but take 20 seconds in Kiev.

Yahoo! and Bing have also identified direct business impacts of page-loading delays as show in Figure 1:

Figure 1 – Business Impacts of Page-Loading Delays

                                                                 Sources: Yahoo!, Bing and Compuware Gomez

Given upfront investment, monthly spending and contract terms, choosing a smartphone represents a $3,000 decision for many consumers. With little or no accurate data upon which to base their decisions, users choose a carrier that appears to suit their mobile footprint and hope for the best, coping with poor coverage, dropped calls and slow data speeds.

While both service assurance and traditional website loading testing and traffic analysis are useful tools, Stratecast views them as incomplete in measuring the 'end user' experience. Consider a dropped a call: chances are that if the wireless network operator had taken a network performance snapshot at the moment the call dropped on the user's handset, no network performance issues would be identified. Likewise, website traffic statistics may present no site performance problems while individual end users are experiencing slow or incomplete page loads, inaccurate page-rendering or other errors.

So there is a disconnect between the 'all clear' signals that wireless network operators and website owners receive from their infrastructure and system evaluation tools versus the performance issues individual end users experience. This disconnect has created dissatisfied users—and a market for new solutions that take an outside-in view of the same sites and services. Application performance management (APM) and customer service assurance (CSA) solutions provide the missing outside-in user perspective as part of a larger customer experience management (CEM) strategy; in this article I'll look closely at APM.

It's Not What You Give, It's What End Users Receive
Virtually every company's customer base is spending more time on the web and increasingly doing so via wireless networks. Whether companies are launching new products and applications or simply trying out various components on their websites, their success in reaching customers and prospects can mean the difference between success and failure as a business.

The business of online retailers, for example, is highly sensitive to web performance issues. E-retailers have a great deal of content on their web pages including photos, both static and animated graphics, and rich media applications such as video. These elements also, however, can increase a page's 'footprint' to the point that it becomes increasingly difficult to download and view on the shopper's device on a consistent basis. If online shoppers cannot quickly access the entire page in a timely fashion, their likelihood to go elsewhere increases.

Yahoo! and Bing have statistically tied page-loading delays to lost traffic and revenues; each additional second of response time above 3.9 seconds per page impacts the business in the following ways:

• Decreases page views by 11 percent;
• Decreases customer satisfaction by 16 percent;
• Decreases conversions by 7 percent.

Beyond these situational impacts, site visitors will equate poor site performance with poor company performance; and, if site issues are particularly severe, it can call into question the viability of the site owner. More broadly, the performance of any site that provides any piece of content or any item for sale from another company impacts that third party as well.

New Imperative: Mobile Performance Should Equal Laptop/Desktop Performance
Mobile websites, apps, games and SMS/MMS initiatives represent a rapidly-growing share of global advertising and marketing expenditures. Yet, as an upcoming Stratecast report on the mobile advertising marketplace will illustrate, forgotten in the hype is the reality that this mobile revolution is built on big expectations on the part of brands (advertisers), agencies, mobile operators and users—everyone with a stake in the mobile services marketplace—that these services perform flawlessly anytime, anywhere. End users, specifically smartphone users, are accelerating the trend because they expect sites and apps to load as quickly on their mobile devices as they do on their laptops or desktops.

If a mobile website, application and SMS initiative fails to perform at speeds that keep up with mobile behavior—standing in line, stopped on a sidewalk or sitting on a bus—it is unlikely that consumers will use the mobile service again. A mobile service that is slow or, worse, down, not only has a negative impact on revenue but increases support costs and damages a company's brand equity through word-of-mouth backlash and social media channels such as Twitter.

To look at it another way, if the Zappos site does not perform up to user expectations, it impacts not only Zappos but Nike and every other footwear retailer placing products for sale on Zappos. It also impacts user perceptions of the mobile operator: it is not the job of users to figure out which link in what APM provider Compuware Gomez terms the application delivery chain (ADC)—is at fault. The user knows this: the service, app or site worked poorly or not at all, it was X's site that failed and it happened over Y's network. In the end, everyone in the ecosystem suffers.

How Do You Provide a Better Experience?
One company providing solutions for optimizing the performance, availability and quality of web and mobile applications is Compuware, via its Gomez web APM platform. Offered on a SaaS/on-demand basis, the Gomez platform provides web load testing, performance management, cross-browser testing and performance business analysis. It provides outside-in, end user experience testing across all end users, browsers, devices, geographies and data centers, using a global network of testers spanning more than 150,000 locations. Compuware Gomez First Mile extends this SaaS-based outside-in view further into the data center, tracking real-user traffic and quantifying the business impact of performance problems by measuring the number of users and pages affected, and thereby isolates the root cause inside or outside the firewall. Gomez counts as customers major portals and online retail/e-tail companies, including 150-200 of the world's top online retailers as determined by media group Internet Retailer.

'Operator Monitoring' for the Masses
Another company, RootMetrics, is truly going where no vendor has gone before. It creates detailed maps of how Verizon, AT&T, Sprint and T-Mobile are performing, neighborhood-by-neighborhood, across the U.S. The company does this by combining crowdsourcing—its CoverageMap app resides on consumer smartphones—and data Root collects itself via Scout Kits in markets across the U.S. While Keynote and Gomez sell to operators and enterprises, as shown in Figure 2, RootMetrics makes the resulting maps available for free (and refreshingly, without requiring a user form fill-in).

Figure 2 - RootMetrics Coverage Map: Austin, Texas, U.S.A., February 17, 9:32am PST

                                                               Sources: RootMetrics and Stratecast

RootMetrics fancies itself as something of a J.D. Power for wireless consumers whose objective is, "To fundamentally provoke a change in the way wireless is measured, bought and sold," just as J.D. Power drove a new consumer focus in the automotive market. Power appears to agree to at least some extent because in December 2010, J. David Power III and James D. (Jamey) Power joined the RootMetrics board. Yet J.D. Power also needs to eat, so RootMetrics is now also publishing reports with in-depth, customized analysis and offering them on a subscription basis to third-party clients—including the four operators that are its current test subjects.

The Payoff: Better User Experiences, Healthier Bottom Lines
By building out global monitoring and testing networks, companies such as these are making available something that every company needs, yet few could afford to build or manage themselves. By showing CSPs and site operators what users are actually experiencing, APM can help them increase revenues, reduce churn and ease the burden on overtaxed help desks/call centers.

Compuware Gomez and RootMetrics are far from alone in this space. Keynote is another leading APM provider, and others include AriesGEO, JDSU, Monitis, Neustar, RoamerNet, Techout and even Mob4Hire, which openly courts testers in a number of social media venues. This is by no means an exhaustive list and also does not include providers Stratecast places in the CSA category.

Jeff Cotrupe manages the Stratecast OSSCS team, which analyzes OSS/BSS, analytics, mobile advertising and new media. He can be reached at jeff.cotrupe@frost.com or +1 760 643 0921.