Is All Research Cooked?

Several weeks ago, I began work on this post. I sent feelers out to a number of people asking if they thought research reports in broadcast and advertising were often manipulated. As I finished a draft of the post and was waiting for a few more responses from my panel members, the Volkswagen scandal broke. So, I deliberately delayed putting this post up until the smoke had cleared a bit.

Mark Twain set the tone for questioning research when he said in the 19th century: “There are three kinds of lies--lies, damned lies and statistics.” Most people say that a clever analyst or executive can spin data in such a way that even poor performance can look much better than it really is.

Responses to my query often had the tone that, “Yes, you can always find a way to lie with statistics.” Now, some of that is not as heinous as you might think. Most of us have been or are in situations where you have to put your best foot forward when research goes against you. A magazine may have a declining audience and circulation but the average income of the remaining readers may have risen smartly. Or, a cable channel that is not growing finds that it does well against certain demographic groups that are highly attractive to advertisers. Where the wheels come off is where people want the research to mirror their pre-existing notions of their brand or media property.

Decades ago, I was an earnest young media analyst who was sent alone to a fairly important client to give a state of the media world presentation. We had no power-points  in those days but I did have a slick deck of acetates and after several rehearsals I faced the music with Mr. Big. The presentation went well and the clients asked many questions and there were frequent nods and smiles as I worked my way through the material. The boss, the marketing chief, told me that he would like to keep slides #2, #17, #31 and the final one. I said of course but wouldn’t you rather take the entire deck. He smiled, said no, and told me that I had a done a very thorough job. Later I found out that he had recommended a course of action to his president and used my four slides to back it up. Of course, by selecting those four slides, and only those four, he had rigged the deck in favor of what he wanted to execute for his brand.

Another time, a year or so later, a client asked me the purpose of research. I said what I still say today--”The purpose of all research is to reduce uncertainty. There is always risk with a new brand and we can never eliminate it but solid research is an important hedge as you begin to market a brand that you are betting millions on.” He seemed to agree and then asked the same question to a contemporary of mine who was the research director of the company. The young fellow said, “Research is the search for truth, sir.” The boss started laughing so hard that he spit his coffee out on the conference table. “That’s rich, son,” was his reply as the young fellow scurried for napkins to clean up the mess.

A year later, the young guy had lost his innocence. I sat in on a research presentation from an outside company. When he was done, the marketing chief thanked the presenter who left. He asked me what I thought and I related how some of it mirrored what was going in media at the time. The young fellow said, “Boss, what do you want me to say in your executive summary to the board?” Somehow, the search for truth was no longer a goal. Both of the players have died so I do not feel horrible telling the story for the first time. Yet, the nagging issue is how much of this type of behavior goes on?

Some of my kitchen cabinet weighed in:

“I don’t think everyone is lying. But, all of us at times try to position things favorably either about our brands or what communications tools to use. It is VERY hard not to have some baggage with you.”

“The big companies are the worst. Remember, 30-40 years ago when the tobacco companies would commission studies saying that smoking was not bad for you? And, I am sure as more evidence comes out about corn syrup that food and cola companies will commission research that will mysteriously exonerate them.”

“When I worked in radio, even if we got killed in an Arbitron ratings sweep, we could always find a few ways to spin the numbers to make the station appear to be first in a few things. Were we crooked? Not really."

A few people wrote that they wonder why some marketing chief and CEO’s even bother to do research. Some get furious looking at customer verbatims about poor service and say, “Impossible. Our customers all love us.” Their attitude often strikes me as “don’t confuse me with facts, my mind is made up.”

When is research the straightest? Some tell me it is when one company is considering buying another. They look carefully for accounting tricks, maybe do customer research, and are brutally honest about how good the company’s product(s) are relative to competition. Once the purchase is made, suddenly key people want to protect themselves or push a specific agenda regardless of what the research is telling them.

So, is all research cooked? No, but much of it is shaded or manipulated a bit.

If you would like to contact Don Cole directly, you may reach him at doncolemedia@gmail.com