Avoiding Crummy Market Research

I came across a great article last week that I wanted to bring to your attention. The article, entitled “Five Ways to Protect Against Crummy Market Research,” was written by Paul Gillin of The CMO Site, and being written from the perspective of an informed respondent, Paul brought up some great points.

Paul begins by pointing out some of the most offensive questions in a survey he recently received. Here’s his favorite (and mine, too):

Envision a scale with Highest Revenue at one end, and Highest Control of the Buyer Relationship on the other. Where would you put yourself on this continuum? (We don’t have the ability to draw a “dial” so please try to balance your answers on this scale.)

It’s painful just reading that question. And it’s hard to imagine that a market research professional was involved in any way in crafting that question. (If they were, they should be fired.)

Paul points out some of the other systemic flaws in this instrument, as well, and what will inevitably happen next:

At least half the questions in this 20-question survey are seriously flawed, ranging from failure to offer an “other” option in a list of choices, to asking respondents to rank relative influence criteria without giving them any way to do so. The results of this research are essentially meaningless.

It won’t matter, though. An executive summary with colorful pie charts paired with the press release can make it look like the Sermon on the Mount — after which, those same charts and graphs make their way into PowerPoint presentations intended to sell something.

This is a problem we face in market research all the time. Anyone can crown themselves a survey author. They may be taking the “DIY” approach (which isn’t necessarily bad, but a fight for another day), they may be unfamiliar with the industry or topics they’re researching, or they may just be bad at their job. Regardless, poorly constructed instruments can have a truly detrimental effect: worse then not getting answers to your questions, you might get answers that are completely misleading, and don’t reflect the reality of the situation you’re trying to understand.

Paul goes on in his article to suggest five steps that those seeking to conduct market research (our customers) should consider before taking on a new project. They’re definitely worth a read.

About Joshua Hoffman

Joshua Hoffman is Technology Specialist at Microsoft and a frequent contributor to Research Access.

Speak Your Mind