Author and humorist Mark Twain allegedly once said, “There are three kinds of lies—lies, damned lies and statistics.” It often gets a laugh despite advances in research over the last 150 years.
I spent a large part of my career working with numbers and still do. Statistics still fascinate me yet, sadly, I see them misused increasingly in the media and more glaringly in social media.
My first real encounter with statistics came when one of my older brothers took a college course in statistics. We discussed it quite a bit and, as a high school student, I bought a copy of HOW TO LIE WITH STATISTICS by Darrell Huff. The book was published in 1954 and by the time I purchased and devoured it, the year was 1966 and it had sold hundreds of thousands of copies.
Huff was cynical about statistics and stressed how marketers could be selective about what you were shown in making decisions. I vividly remember one effective example in the book. He talked of a survey taken in 1950 regarding the income of Yale graduates in the class of 1924. The average income reported was several times that of the national average. One might say no surprise as many Yale graduates were and are among America’s elite. Huff probed a bit—what of those who did not respond? Were they struggling compared to their classmates? Embarrassed by their relative lack of financial success? This was my first exposure to non-response error, and he did a better job explaining it than any college or graduate school professor that I ever had.
In college, I did fine in Statistics and firmly understood the difference between average and median which many in the media and business world still do not grasp or worse, use the two terms interchangeably.
In the advertising and marketing world where I spent the bulk of my career, the response to any research presented was varied. A wealthy businessman who owned 20 fast food restaurants chewed out a young and earnest member of my negotiation team by saying her buy was worthless as he did not watch the programs on the TV schedule. I had had enough and told him that he was a multi-millionaire, and it was logical that he was watching vastly different programs than his 18-34 target audience. He got red in the face, but his fellow members of the advertising co-op backed me up. Sales moved up nicely and he stayed quiet at the next meeting.
Other clients told me that all Nielsen ratings were nonsense as they had never been in a survey. A primer on sampling theory would have accomplished little.
Over the last 50 years, I have seen good, mediocre and poor studies. Here are a few things to keep in mind:
1) When you see the research, take your own temperature. How do you react to it emotionally? If it is a totally new topic, you are likely to be cool-headed. I have seen CEO’s or senior management bristle when research says consumers have issues with their product or service. “All our customers love us” was a line heard frequently yet their market share was not high. A new CEO was likely to be much more open to hearing bad news than their predecessors as they were there to turn things around. I have seen the same thing with shareholders. If an analyst posts a less than positive forecast for a stock that they own, often the response is that the category specialist is an idiot.
2) Try to separate your personal feeling from what the larger sample is telling you. I saw broadcasters dismiss cable and then digital possibilities as they felt their growth would hurt their livelihood. They were not objective at all.
3) Ask if anything is missing from the data and how was it asked. Most do not.
4) I realize this is hard for all of us but try hard to keep an open mind and remain curious. Most people are not, and this is deadly in our era of rapid change.
I could not on forever and provide some colorful examples of reaction to research, but I think it best if I take those stories to the grave.
If you would like to contact Don Cole directly, you may reach him at doncolemedia@gmail.com or leave a message on the blog.
No comments:
Post a Comment