Have you ever struggled to understand why your ad is performing the way it is? Or better yet, which part of the ad is driving its result – and how you can duplicate (or avoid) it in future development efforts? If so, then we have the perfect solution, and it utilizes our validated CCPersuasion metric that has been proven to predict in-market Marketing Mix Modeling results.

Our solution is an exercise we call Scene-To-Scene, which is available in our Touchpoint Plus methodology.  It involves splitting the ad into its constituent “scenes” and then gauging positive and negative interest for each of them.  By taking this data and crossing it with our CCPersuasion exercise, we are then able to establish which parts of the ad are actually having the most impact on the ad’s effectiveness – both positive and negative.

To show this firsthand, we have included an example for a recently aired DirecTV NOW commercial that ran heavily during the Spring 2018 months. If you watched a lot of March Madness coverage, then you may very well remember seeing this ad. Here is a snapshot along with the ten “scenes” we established for it:

 

The first four scenes feature the opening drama, which parallels with the idea of switching from more traditional cable to DirecTV NOW – it includes an actress portraying a girlfriend who is throwing her soon-to-be ex-boyfriend’s belongings (including his traditional cable box) onto the front sidewalk.  Each “scene” includes a line from the ad so we are able to assess them individually – as well as collectively – if desired.

Beginning with scene 5, the next handful of scenes each talk one specific benefit of DirecTV NOW, which we are also able to assess individually using the Scene-To-Scene approach:

• Scene 5: Live TV
• Scene 6: No hardware
• Scene 7: No satellite
• Scene 8: No contract
• Scene 9: $10/month for 3 months

Scene 10 is an ending tag with the call-to-action to the website.  While we do not necessarily expect the tag to generate a ton of interest, this allows us to check and make sure that is the case.

How did it turn out? See for yourself here:

 

As we expected, the opening drama in the first half of the spot received very polarizing responses – with a couple scenes actually receiving more negative interest (red) than positive interest (green) agreement, such as scenes 1 and 3. Others are netting more total attention, but still with substantial negative reactions, such as scenes 2 and 4.

 

The second half of the spot – well, that’s a different story:

 

Scenes 8 and 9, which talk “no annual contract” and “$10 a month for 3 months,” respectively, receive by far the most favorable responses of any individual scene in this ad.

However, while this is an obvious qualitative win for those two scenes, which are heavy on the product messaging and not so much on the creative drama, it does not necessarily prove that these scenes themselves are great at driving the ad’s CCPersuasion level…

 

Therefore, we can cross these responses with CCPersuasion responses and approximate just how impactful these scenes are – as well as all the others. See this in the graphic below, which includes a % below each scene indicating its share of impact on the CCPersuasion result:

 

As we likely expected, this verifies scenes 8 and 9 are powerful in driving the persuasive power of this spot – combining for half of the result and being far and away the most impactful scenes.

However, this also shows data we may not have expected with the opening drama. Scenes 1, 2, and 4 – which all involve the girlfriend actively “throwing out the old” – all return around 10% of this ad’s persuasion power apiece – meaning that even though they were not necessarily receiving significantly higher positive responses than negative responses, they are still valuable to the spot – especially setting up interest in the product benefits.

Scene 3, which appeared to be similar to Scene 1 on the qualitative scale, is actually not nearly as powerful in driving viewers to DirecTV NOW. This scene is different in that it is a break from the action and the “sorry, not sorry” line is also a break from the definitive change message of the other opening scenes.

Scene 5, which talks “live TV,” does not appear to be very powerful either qualitatively or in driving CCPersuasion, but perhaps is a necessary set-up line to explain what DirecTV Now is, as opposed to on-demand streaming services such as Netflix.

And while the “no bulky hardware” and “no satellite” scenes achieved fairly decent qualitative feedback, we can see that they really do not compare to the two scenes that come directly afterward (8 and 9) in terms of driving CCPersuasion.

 

Last, but not least: we are also able to provide qualitative feedback to add perspective into what exactly is driving each of these scenes. Every study that includes Scene-To-Scene will also include a file that shows what exactly respondents are saying drove (and didn’t drive) their interest. Here are some examples for one of the most popular scenes in this spot (scene 8 – “no satellite”):

• “I have satellite now, and would like to try something different.”
• “It would be nice to not need a satellite”
• “I like that you don’t need a satellite.”
• “I like the idea of no satellite which often goes out due to weather conditions”
• “This means that there is no need to drill or screw or ask landlords/HOA permission to subscribe to a satellite dish.”
• “Because when I think DirecTV in general I think of Satellite and this doesn’t need one”
• “Because DirecTV has been a satellite focused company in the past”
• “You do not have to have an unsightly piece of hardware attached to your roof”

 

Thank you for reading.  We hope you enjoyed this blog entry – and please look out for more in the future.

 

For more information on how our communications research tools can help improve your advertising’s effectiveness, please contact us at aklein@msw-ars.com.

The use of technology and social media in qualitative research over the past few years has undoubtedly been a welcome addition to the researcher’s toolbox.  Indeed, we’re a huge proponent and regularly incorporate such methods into our projects. However, it’s crucial to not let the buzz over technology and social media overshadow the need for solid research design, or to diminish the value that can be gained through more traditional qualitative approaches.  Indeed, in spite of all the hype over the use of technology, the 2014 GreenBook Research Industry Trends Report indicates that the most widely used qualitative methods continue to be in-person focus groups, in-person IDI’s and telephone IDIs; with no significant change in use noted among any type of qualitative method from the previous year.  So we are not advocating the use of one approach over the other, but rather stating the case that there is a place for all of these tools, as long as they are being used for the right reasons.

1)     Rather than pitting technology against traditional methods, or viewing these varied approaches as an either/or proposition, consider all approaches as viable and complementary

• An increasing amount of qualitative consists of a hybrid of methods vs. a singular approach
• We encourage experimenting with new methods in order to better understand the value they offer
• There’s always room for new tools, it’s a matter of knowing how and when to use them

2)     Make certain that the research objectives and requirements are driving the methodology… not the other way around

• Remember, garbage in-garbage out
• Fairly weigh the pros and cons of each approach, and determine which will best suite the research, as opposed to force fitting a method for no other reason than its novelty or newness

3)     Above all, don’t overlook the skill set required of the qualitative consultants conducting the research, regardless of the chosen method

• The need for a solid foundation and understanding of qualitative design
• The mindset that, “anyone can conduct focus groups,” is as false for technology-based methods as it is for traditional approaches
• The importance of asking the right questions and knowing how to listen
• The ability to extract valuable insights is where the true value lies
• Working with researchers who understand people more than technology

Following are a few examples which illustrate how technology and traditional methods can co-exist, and when one approach may offer advantages over the other:

Mobile Ethnography:  Mobile technology allows participants to self-report “in the moment,” communicating via any combination of text, audio and video from whatever environment the research calls for.  While this method may offer the benefit of capturing a person’s feelings and experiences in-situ, there may be other behaviors or actions that the respondent is not capturing or reporting, or possibly not even aware of, that a trained ethnographer would notice.  Thus, immediacy and speed may be gained at the loss of small, but very telling details; as what people don’t tell and don’t do can be some of the most valuable information gained in an ethnographic project.

We might suggest that a combination of shopping trips with and without a researcher present may provide balance, as learning gained on the assisted trips may help explain behaviors noted on the self-reported ones.  Or, self-reported trips followed by either in-person or webcam interviews allowing for further probing and exploration might also be considered.

Focus Groups/IDI’s:  Both face-to-face and online interviews (real time or bulletin board) have their pros and cons.  Deciding which route to follow may be dependent on a number of factors–such as the ease/difficulty of recruiting qualified respondents, budget and time constraints, geography—but the key determinant should be the objectives of the research.

A relatively simple, straightforward concept screen or evaluation can easily be handled by any number of online platforms offering markup tools, which allow participants to view, critique and comment on concepts without being influenced by others, and then allow for discussion.  However, a project more exploratory in nature, where the sharing and building of ideas is important, or having people with different views challenge each other and engage in more natural flowing conversation is critical, would better lend itself to face-to-face groups.

While both of these scenarios could be handled through either method, for the latter example the benefit of being able to read non-verbal cues such as facial expressions or body language, and hear voice intonation would favor a more traditional group approach.

Social Media:  Social media is too big to be ignored as a source of real-time information for companies and researchers, but caution needs to be taken in terms of how it is used as a qualitative tool and its influence on decision making.  One of the primary concerns is not knowing enough about the people providing commentary, or having the opinions of few speak for many.  Some research conducted on social media shows that the majority of people use it for consumption without being active contributors in sharing content or interacting with others, and while there may still be valid learning gained from those who do participate, researchers need to be aware of the bias that exists.  And while there are numerous text aggregation and analytic tools that troll popular social sites, is there one that can truly interpret language nuances and understand the context in which comments are made–incomplete and grammatically incorrect sentences, sarcasm, humor, anger, irony?

As this early stage, social media content may be most valuable in providing fodder for further qualitative exploration… through either tech or traditional methods:  developing hypotheses, identifying language used around brands or categories, bringing potential problems with products or services to light, and raising other questions that may prompt meaningful discussion on key issues.  However, at this early stage, we advise caution in viewing social media research as a stand-alone qualitative tool.

MSW●ARS recognizes the value technology offers, and is continually experimenting with new methods and tools, without losing sight of the value brought by tried and true qualitative approaches.  Please give us a call to discuss your qualitative needs, and allow us to recommend an approach that utilizes the best of both worlds.