Why I’ve Lost Faith In Quantitative Research For Competitive Benchmarking
By
Randy Shattuck

It seems pretty much every day now I see an email or post on LinkedIn that invites me to compare my marketing plan, budget or approach to “best-in-class” companies. I used to pay attention to these and even put some stock in them. In fact, I’ve built marketing plans in the past based in part by what these surveys say.

But no longer. I’ve lost faith in most quantitative research studies that provide competitive benchmarks. I think you should too. Here are my reasons.

My top reasons

Here are my top four reasons for having lost faith in most quantitative research studies for competitive benchmarking.

Let’s look at each of these in more detail.

Only companies whose primary business is research practice sophisticated enough models and intellectual rigor to produce credible data

So before I go too far on this subject, I want to qualify a few things. I do believe in the value of competitive benchmarking research if it is based on credible data and professional research methodologies from an independent third-party.

I don’t believe you should blindly follow whatever competitors are doing. But I do think it’s valuable to know what competitors in your industry are doing to go to market, win deals and position effectively… If you can trust the data.

But that’s really the core of my issue. So much of the so-called competitive benchmarking data that is being presented these days is based on specious and unreliable research methods. In part, this trend has come about because online research tools have made it easy. Anyone can sign up for a SurveyMonkey account and begin asking questions within minutes. But will the data be reliable, useful or insightful? I don’t think so.

Let me tell you why I say this. I have a background in psychology and statistical analyses. I believe in being very data-driven, as my clients will attest. But I believe in good data, sound data that is related to the topics and issues that will help me and my clients.

There are organizations today who practice methodological rigor, who have core expertise in research techniques that produce accurate information. Many of these organizations have a core business model in research. They build their reputation around the value of the research they produce. They have people who are trained in how to develop studies that produce reliable and accurate data.

But it seems to me that the majority of the so-called research studies I’m hearing about do not come from these organizations.

The studies I am seeing come from companies with a vested position in realizing certain outcomes. Here is what I mean. Do you remember that television commercial that said – 9 out of 10 dentists surveyed prescribe this gum for their patients who chew gum? My question was always this: who picked the dentists to be in the survey?

In other words, if a company wants to sell you a given product or service and their research findings indicate a strong need for that product or service – can you really trust that research? Wouldn’t it be shocking if their research showed that you didn’t need the products or services they offer? Hmmm… Kind of makes you wonder doesn’t it?

How research teams interpret and manipulate data

I’ve been a part of some pretty large research studies in the past that arise out of a few different industries. One of the big challenges that always occurs after the primary source data is collected is how to interpret the data. As Mark Twain said, there are lies, damn lies and statistics.

I have seen some significant mental gymnastics taking place when research data comes back that does not support the original hypotheses. I have watched as research outliers are discarded and the data is sliced and diced to make studies report findings that are… Well let’s just say the finding didn’t arise out of an objective review of the data.

This is what happens when research studies are conducted by organizations who have a vested interest in seeing the data arrive at certain conclusions. I’m not saying it always happens, but I am saying I have seen enough of this to know that it does happen.

I find most research methodologies to be specious

Here is something I do, and I encourage you to do this too, when I see a new study come out that sounds interesting. I will access the study and then look for the research methodology that gave rise to the data. I want to know if the study was qualitative or quantitative. I want to know how research participants were chosen. I look for the N signifier, which usually tells you the number of people who participated in the study.

If these data points are missing or if the methodology is not revealed, I question the validity of the study. I think you should too.

I don’t believe it’s wise to base your marketing plans on what your competitors do

But none of this quite gets to the heart of the matter for me. Let’s assume for just a moment that you have on your desk a very valid study, based on credible research methods that tell you exactly how your competitors are going to market. What do you do with that data?

I do not believe it is wise to craft marketing plans that mirror what your competitors are doing. Do you really only want to be as good as your competitors? One of my major issues with these types of studies is that they seek to be normative. They seek to establish the average of what everyone is doing. I don’t know about you, but I’m not looking to be average. I’m striving for excellence.

If you want to stand apart, carve out your own space and really advance your company, it’s good to know what competitors are doing but you probably need to do other things. You probably need to try other strategies and tactics.

A much better way

If you want to build marketing and sales plans that deliver excellence, competitive advantage and differentiation, I have a suggestion for you. This is what we are doing and it works amazingly well for our clients too.

If you can uncover these three attributes, you can really accelerate your success. Why are these three the most important?

Qualitative research should reveal the top goals, opportunities and challenges of ideal clients. These topics give you the insights you need to create content marketing programs that people like your current clients will not be able to resist. This content should be the substance of what they consume on their in-bound journey to you. This is one key way to replicate your current ideal clients.

The qualitative research should also reveal where your people who fit your ideal client profile tend to aggregate. For instances, architects may discover the associations and groups that regional and city planners have joined and their associated conferences. Consulting firms may discover the groups that people who fit their ideal client profile tend to join on LinkedIn. Financial advisors may discover the networks and associations that affluent people who fit their ideal client profile tend to participate in.

Finally, qualitative research should also indicate how people discover and consume information when they are looking for a service provider like you. Content marketing is changing the game for professional service firms who prefer not to wait on referrals, the classic method for new client acquisition. But not all content assets or content types are equally effective. One of the biggest challenges you’ll encounter in developing content is the choice of format.

For instance, many service firms these days have a blog site. But what if you discovered that most people who fit your ideal client profile do not read blog sites? How would that change the way you go to market? What if you discovered that most prospective ideal clients, for example, prefer video to written content? Would you put more resources behind video?

If you understand the topics that matter to ideal clients, understand how to get your content assets in front of them and know that they will consume those assets, your pipeline will get very full.

How to get started

To help you move in this direction, I’ve created a webinar called Nurture Leads To Fill Your Sales Funnel: How Professional Service Firms Can Realize Consistent, Predictable Growth. This free resource shows you how to build a lead nurturing plan filled with content that prospective ideal clients will love.

Randy Shattuck is a senior marketing executive, marketing automation guru and founder of The Shattuck Group, a full-service marketing firm. Using 8 core services, The Shattuck Group empowers clients to reach their next level.