Recently I was listening to NPR Freakonomics radio in their “Want to Think Like a Freak?” series. In one segment Steve Levitt, the co-author of Freakonomics and lauded economist talks about his consulting firm and tells a story about when he worked with a well-known national department store to help with their advertising strategy. This store’s strategy was to run two types of advertisements; print ads that ran every day in over 200 markets and then television ads three times a year, before Black Friday, Christmas and Fathers Day. They had been doing the same thing for years and their sales data told them that TV ads were four times more effective than their print ads. However, Levitt realized that this company seemed to be spending hundreds of millions of dollars on advertisements to try to persuade people to go shopping at a time they likely were already planning to go shopping. This is indicative of complementary advertising. Levitt found their assumption based on sales data very odd and naturally as an economist wanted to run an experiment. He told them that they can’t learn the truth with the existing data and wanted to pull print ads from 20 markets for a 3 month period to see what happens to sales. The response from the client? “Absolutely not, we can’t just not advertise, our CEO will kill us.” In this same conversation the executive at the company mentioned once having an intern who was supposed to do the ordering of the ads for the Pittsburg area but he forgot to order them for the entire summer. “I hope you hired that guy” said Levitt.
Essentially the intern had done an unintentional natural experiment for the company. Because of this mix up they were able to examine the data from the months of missing advertisements and found – wait for it- that there was absolutely no impact whatsoever on sales in the Pittsburgh area.
Now this could have been for a few different reasons. There is first the question of if the advertising was informative, persuasive, complementary or a combination of them all. Advertising is able to serve as a barrier to entry for other similar companies when it is persuasive in nature. The advertisements could also have been informative because when they stopped there was virtually no change in sales. Empirical studies show that brand equity only lasts about three months and these ads were stopped for about 3 months. Based on these findings the best thing for the company to do would have been to submit to a longer, more extensive and monitored study. Based on these options it is likely that the advertisements do have an impact yet the company has a strong brand following so the experiment was not long enough to see any significant results.