Copy testing has been utilized by advertisers for decades to assess the quality of advertising copy. MSW’s TouchPoint™ copy testing system has been extensively validated, showing that test results on key metrics are predictive of subsequent sales results from airing the tested advertising. A partnership between MSW and predictive marketing analytics firm Keen set out to assess the utility of test scores from MSW’s Insight Rabbit DIY copy-testing platform at improving predictions from Keen’s MIDA decision support system.

The MIDA (Marketing Investment Decision Analysis) platform is designed to help marketers decide how to invest in marketing activities. MIDA users can develop optimized investment scenarios to meet specific business objectives such as hitting revenue targets or meeting budget constraints. It does this by applying a Bayesian modeling approach to a wide range of a brand’s historical marketing and performance data.

Could the use of copy quality metrics improve forecasting of business outcomes and hence, be used as an input to MIDI to improve the allocation of marketing dollars? To address this question, new advertising for a major packaged food brand was selected. This brand had developed two different campaigns with different communication objectives that tied back to the brand’s strategy. The brand intended to air both campaigns concurrently.

The television ads developed for each of the two campaigns were tested using MSW Research’s Insight Rabbit Pulse Lite copy testing solution. Results are shown in the graph below.

Both ads were adequate in terms of the secondary Break Through metric which assesses the degree to which an ad leaves viewers with a memorable and branded impression. However, Copy A scored much stronger in terms of the CCPersuasion™ metric which assesses the degree to which the ad positively influences preference for the advertised brand. Prior validation studies have shown CCPersuasion to be the strongest predictor of an ad’s selling power. Copy A scored significantly above the Fair Share benchmark with an index of 161, suggesting it is a very strong piece of copy. On the other hand, Copy B indexed 113 versus the norm and would be considered slightly above average at best.

Historical performance of the brand’s television investment was measured in MIDA to quantify the expected returns on investment for an average (or benchmark) ad for the brand. Then an initial forecast was developed before the start of the campaign using this historical performance enhanced by the MSW copy test results along with planned media delivery levels.

After the campaign had been running for six months, MIDA was updated with actual sales and television campaign delivery data. As seen below, the ROI for Copy A was approximately 90% higher than would have been expected from the historical benchmark ad performance level. On the other hand, Copy B’s ROI was only about 5% higher than the benchmark expectation.

This actual performance was extremely consistent with the copy test results which suggested that Copy A was a very strong ad, and that copy B was slightly above average. This result illustrates the utility of MSW copy test scores in the a priori forecasting of investment levels through integration with decision support systems. The integration of MSW copy test scores with a decision support system like MIDA would help steer marketing dollars toward more deserving initiatives, improve forecasts and bolster in-market effectiveness of brands’ marketing programs.

More than 100 years ago, John Wannamaker was famously quoted as saying: “Half the money I spend on advertising is wasted; the trouble is, I don’t know which half.”

Only half?

Fast forward 100 years, and many marketers still don’t know how much of their advertising is wasted. And with the myriad new methods for advertising (e.g., digital, social media, reviews, etc.), we’d be lucky if we are only wasting half our money. John Wannamaker would be horrified!

That is why it is so critically important for marketers to use validated, sales predictive metrics when evaluating advertising. Validation means that the metric predicts real, in-market behavior: brand preference, sales, share lift, revenue, and loyalty. Advertisers who do not use market proven validated metrics risk wasting significant resources both in advertising expenditure and in the health of their brand.

Validation Obsession

MSW is obsessed with validation and has been for decades. As far back as the 1970’s, MSW began testing their fundamental metrics for predictability. Since then, MSW’s validation efforts have been continuous and rigorous, including cross-client analysis, client sponsored independent studies, publishing in academic journals, and independent industry trade group audits. Here are a few examples regarding MSW’s CCPersuasion metric (which measures a pre-to-post change in brand preference as the result of advertising):

• L’Oreal: BBDO’s conducted an independent analysis for three L’Oreal brands: Preference, Excellence, and Revitalift. Their analysis concluded that MSW metrics strongly validated with advertising in-market performance. And as determined by L’Oreal a one-point increase in CCPersuasion yielded a 0.6% lift in retail sales. The analysis also determined that as CCPersuasion increased versus the norm, so did Annualized Customer Value (ACV) with a top 10% television ad resulting in an increase of ACV of $42.5 million.
• Celestial Seasonings: Celestial Seasonings evaluated five selling propositions, moving forward with the strongest which measured CCPersuasion significantly above Fair Share. The brand went to air with an ad that outstripped even that superior selling proposition. “. . . Our advertising didn’t just steal share from competitors; it grew the herbal tea category. While tonnage for most other hot beverages was declining during this time period, herbal tea sales were up, with increases as high as 12 percent in one of the spot television markets,” said VP Marketing Mike Mondello.
• StarKist Tuna in a Pouch: Our research suggested increased emphasis on the “fresher and better taste” product benefit could improve effectiveness. In fact, the finished execution scored 2.5-times the benchmark. “The results from the initial advertising quarter yielded an ROI of 76 percent, an enormous improvement over the break-even ROI we had expected for the quarter using a traditional approach,” reported Barry Shepard, StarKist.
• Walmart: Walmart compared the incremental sales generated (from MSW’s Marketing Mix Modeling) by a specific television ad to the CCPersuasion points delivered (the CCPersuasion score combined with media weight) for that ad. The strong relationship (r = 0.82) confirms CCPersuasion measure is a valid predicter of the sales effectiveness for advertising for Walmart.

CC Persuasion Audit Summary

A combination of industry, agency, and academic leaders created the Marketing Accountability Standards Board (MASB) to ensure“ the efficient and effective functioning of a marketing-driven business because decisions about the allocation of resources rely heavily on credible, valid, transparent, and understandable information.” MASB then established their Marketing Metric Audit Protocol (MMAP) to validate whether marketing and advertising research metrics accurately measure and predict the market behavior they purport to predict.

MASB conducted an 18-month study across 12 product categories and 120 brands, comparing MSW’s CCPersuasion metric to Unit Share. Not only did MASB confirm the strong relationship between CCPersusion and Unit Share (r = 0.88), but it also found that accounting for price and distribution improved the relationship to an almost unheard of 0.94. The MMAP audit concluded that “the CCPersuasion metric is able to quantify the likelihood and magnitude of an ad’s impact on future sales volume and market share.” MASB also concluded CCPreference has been “proven to reflect market share with a high degree of accuracy across a spectrum of diverse categories.”

Summary

As Gayle Fuguitt, former President and CEO of The Advertising Research Foundation summarized, “It is the most complex time in the history of advertising, yet the C-suite questions are simple:

• “Do my marketing dollars drive growth and sales?”
• “Where do I spend the next marketing dollars?”

MASB’s work, focused on the accurate and comparable measurement of marketing efforts, is advancing the industry to respond to these C-suite needs and to drive much needed growth.”

However, MSW is not resting on their laurels. Their validation obsession drives them to continually evaluate and test their metrics to ensure in-market predictability.

Does your advertising drive business results? Do your advertising metrics accurately predict success? Not sure? Contact us today