Benchmarking Comes off the Bench

When you hear the term “benchmarking” you might think about a row of servers or other computer hardware being pitted against each other to see which processor is fastest. But instead of just determining who’s the best and greatest, benchmarking goes way beyond that to give you the means for improving business effectiveness through critical analysis of data.

If you look specifically at the communications industry, today more than ever service providers can’t take anything for granted. Even if you’ve got the biggest market share, we all know how things can change in a heartbeat, and before you know it you can find yourself at the bottom of the heap.

Business benchmarking is just one tool in the arsenal for providers interested in maintaining profitability, hanging onto customers and pushing forward with new services and initiatives, but it takes on a whole new angle in today’s economic climate.

TM Forum launched our Business Benchmarking program about five years ago, and honestly we thought it would be an immediate success in the industry. But what we hadn’t realized is even with a great idea like benchmarking for service providers, it’s one of those things where “success breeds success” and it gradually builds on itself year after year.

So in our first year of running the program, we had convinced 10 or so service providers to put their data in, but it was an uphill battle. We had to explain to each new provider the concept of benchmarking, what we are trying to achieve, and why they needed to part of it.

But as anyone who’s been in this business for a while knows, in communications everyone loves to follow the leader, and no one likes being too far out on a limb. So it took a while to ramp up on our service provider participation. But ramp up it has. By our third year of doing benchmarking studies, we had about 40 providers participating, and in the past 12 months we’ve gone from 40 to well over 100.

Studying the Industry
Each year we run a number of different benchmarking studies, with varying levels of participation. I’d say 15 to 20 providers for a single study is a great number, but some studies are seeing 30 or 40 participants.

Our topics run the gamut, so there’s going to be something of interest to just about anyone. We have studies on broadband that delve into general business performance and more specialized areas like order-to-cash. We also do mobile studies that cover areas like mobile broadband and mobile business performance. We also look at high-speed business services, which study provisioning of IP services and IP network performance. Also on the list are areas under revenue management, such as billing performance and revenue assurance.

Now, I know as well as anyone that there are companies out there that do commercial benchmarking for service providers, but often the cost to participate in one of those endeavors can run into the millions of dollars. Our cost? Absolutely nothing to TM Forum’s service provider members. I’d say the breadth of our program rivals and probably exceeds the commercial efforts, and that’s what makes it such a valuable member benefit to providers.

It’s clear we’ve finally achieved critical mass, and that a high level of participation only adds more credibility to the benchmarking results. Speaking of which, our reports are tailored for each provider, showing where they fall among all providers in the survey. In our secure database, we can also drill down and show providers how they rank compared to other operators of their size and within the same geography. We can basically slice and dice the data any which way you can think of to give meaningful results to all participants - all the while protecting the service provider’s identity.

It should go without saying that anonymity is assured with all studies since that’s the only way providers will be willing to part with their sensitive details. And while vendors and system integrators can purchase subscriptions to our benchmarking data, they can only see general distributions and details. They never see any information that would lead to attribution.

Benchmarking and Standards: Synergy in Action
Besides giving providers a clear understanding of how they stack up against their peers and competitors, our benchmarking program goes even further and has actually evolved to a point where it goes beyond pure data collection and analysis. Our program is now an entry and exit point for a lot of the technical work that goes on within TM Forum.

What we’ve traditionally done is define key measurement points and talk about best practices when it came to a particular standards area. But because our benchmarking program is becoming so popular, it’s actually changing where we place our priorities. For example, one of our technical programs may help define essential metrics for the benchmarking program, so it can run a real live study as defined by our Collaboration Program. Then the results of that study can feed into driving our collaborative work and areas where best practices need to be defined.

We can then rerun the benchmark against the same metrics and see how we’re improving and how service providers are improving against those key metrics. By bringing two key areas together – benchmarking and standards – TM Forum is in a unique position to provide this important leadership to the communications industry.

By broadening the reach of benchmarking, everyone succeeds in the end. And during slower markets, anything we can do to improve business processes and efficiencies for service providers will go a long way toward future achievements.