L36: Pre-testing & Post-testing Methods

Good morning, everyone. Welcome to Lecture 36 of MGA-304. Last class we covered the advertising testing process — the stages from strategic research through concept testing to finished commercial testing. Today we go into the specific methods used at each stage in more detail: Pre-testing and Post-testing Methods. This is technical material but extremely important for anyone who will be evaluating advertising research in their careers. [0–10 min: Introduction] Let me start with a principle that should guide all thinking about advertising research: the purpose of research is to reduce decision-making uncertainty, not to eliminate it. No research method will tell you with certainty whether an advertisement will be effective. Every method has limitations, every sample has biases, and consumer behaviour in research settings differs from behaviour in the real world. The goal is to reduce the chance of catastrophic failures and to learn as much as possible about how consumers are responding to the communication before committing to expensive media expenditure. With that principle in mind, let us examine the specific methods. [10–40 min: Core Content] Pre-testing methods are those applied before the advertisement goes to air or is widely distributed — typically after creative development is complete or nearly complete. Method one: Focus Groups. Qualitative, small sample, in-depth discussion. Six to ten consumers from the target audience discuss their reactions to advertising stimuli with a trained moderator. Advantages: rich, nuanced, exploratory — you can probe unexpected reactions, understand the language consumers use to describe their experience, and identify issues that structured questionnaires would never find. Disadvantages: small samples mean results are not statistically projectable; group dynamics can distort individual responses — dominant personalities can influence quieter members; consumers may not be honest in a group setting about reactions that feel embarrassing or socially undesirable. In India, focus groups are extensively used by agencies and clients at the concept testing stage. The richness of qualitative feedback from an Indian focus group can reveal culturally specific issues — a visual that has unintended caste connotations, a music choice that is associated with a competitor, a language register that feels inauthentic for the target demographic. These issues can only be identified through deep qualitative exploration. Method two: In-Depth Interviews (IDIs). One-on-one conversations between a researcher and a consumer. More expensive per respondent than focus groups but free from group dynamics. IDIs are preferred when the subject matter is personal or sensitive — financial products, health conditions, or when the researcher needs to probe individual psychology deeply without social pressure. Method three: Survey-Based Quantitative Pre-tests. Large sample, structured questionnaires measuring specific advertising metrics. The standard format: the respondent is exposed to the advertisement in a controlled setting — on a screen, in a mock magazine, or embedded within other content — and then answers questions about: Recall, Communication, Liking, Brand Linkage, and Purchase Intent. The structured survey allows statistical analysis and comparison against norms. Disadvantages: structured questions can only measure what you think to ask about; forced-exposure testing does not mimic natural media exposure where the consumer has the option to ignore the advertisement. Method four: Theatre Testing or Clutter Testing. The advertisement is embedded within a 'reel' of other advertisements and programming content, and shown to a group of consumers in a theatre or viewing facility. This mimics natural clutter conditions more realistically than isolated exposure testing. The test measures: Whether the advertisement broke through the clutter and was noticed. What was recalled immediately after. Attitude and purchase intent measures before and after exposure — the difference indicating the advertisement's effect. Method five: Eye Tracking and Biometric Pre-tests. As discussed in Lecture 34, these physiological measures are used to assess visual attention patterns and emotional engagement with advertisements. Eye tracking for print and digital advertising tells you exactly which elements are looked at, in what order, and for how long. Biometric measures capture moment-by-moment emotional engagement during a television commercial. These methods are particularly valuable for identifying specific problem areas within an advertisement — a visual that confuses rather than attracts, a scene that generates disengagement. Method six: Neurometric Testing. EEG and fMRI-based testing measuring brain activity during ad exposure. The most sophisticated and expensive pre-test method. Capable of measuring attention, emotional engagement, and memory encoding simultaneously. Used by major global brands for high-stakes creative decisions. Beginning to be used in India by major FMCG multinationals. Now let us examine Post-testing Methods — research conducted after the campaign has aired, measuring its impact in the natural media environment. Method one: Day-After Recall Testing. Traditionally, research interviewers contacted consumers the day after a major television commercial aired and asked whether they recalled seeing advertising for the brand. Those who recalled were asked to describe what they remembered. The recall score — the percentage who could recall — was compared against category norms. Day-after recall was the dominant copy testing metric in the USA and India through the 1970s to 1990s. Its limitations: it measures only whether the advertisement was remembered the next day, not whether it built brand equity or generated purchase intent over the campaign period. Method two: Continuous Brand Tracking Studies. This is the most widely used post-testing approach among large advertisers in India today. Research is conducted on a rolling basis — weekly or monthly surveys with matched samples from the target audience — measuring: brand awareness, advertising awareness, brand attribute perceptions, consideration, and purchase intent. Continuous tracking allows brands to monitor the effect of a campaign in real time and to detect when communication effects fade, signalling the need for refreshed creative or increased media weight. HUL, P&G India, and Mondelez use continuous tracking for all their major brands in India. Method three: Campaign Evaluation Studies. A specific pre-post research study designed around a single campaign. Pre-campaign benchmark measurements are taken before the campaign, post-campaign measurements after. The difference represents the campaign's communication effect. These are conducted for major campaign launches or when a brand has made a strategic change in creative direction that needs validation. Method four: Sales Response Modelling. Uses Marketing Mix Modelling to isolate the sales contribution of the advertising campaign from other factors. As discussed, this captures short-term sales response well but under-represents long-term brand equity effects. Method five: Digital Analytics Post-testing. For digital campaigns, comprehensive performance data is available through platform analytics — Google Analytics, Meta Ads Manager, and third-party measurement platforms. Metrics include: impressions, click-through rate, video view-through rate, cost per acquisition, return on ad spend. In-app tracking for brands with direct e-commerce provides conversion data all the way to purchase. A crucial concept is the distinction between Claimed Recall and Actual Recall. In research, when consumers are asked 'have you seen an advertisement for Cadbury recently,' they may say yes even when they have not — a phenomenon called false recall or brand confusion. Conversely, they may have been exposed and influenced by advertising but not consciously remember seeing it — what is called 'hidden' or 'implicit' advertising effects. The standard recall methodology measures only conscious, explicit recall. Implicit effects — the way advertising builds brand associations in memory below conscious awareness — are harder to measure and may represent a significant portion of advertising's actual effectiveness. [40–55 min: Activity and Discussion] Let us work through a scenario. Tanishq runs a new television and digital campaign for three months. Post-campaign research shows the following compared to pre-campaign baseline. Unaided brand awareness: unchanged at 72%. Advertising awareness: 58% recall seeing Tanishq advertising in the past month (up from 40% at baseline). Message association: 65% of those aware of the advertising associate it with 'Tanishq celebrates modern Indian women' (up from 45%). Brand attribute 'aspirational': 55% rate Tanishq as aspirational (up from 47%). Purchase intent: 38% say they are likely to purchase Tanishq jewellery in the next three months (up from 33%). Sales: no significant change in Tanishq's reported market share for the period. Evaluate the campaign's effectiveness using these post-test results. What are the strengths of the campaign? What concerns would you raise? Discussion: Strengths — strong advertising awareness gain confirms media delivery and creative cut-through. Message association improved significantly — the communication of the strategic platform is working. Brand attribute improvement on 'aspirational' confirms the desired attitude change. Purchase intent improved modestly. Concerns — no sales response yet, but this may reflect the lag between attitude change and purchase — jewellery is a high-involvement, low-frequency purchase category. Also, unaided awareness was already very high and movement was not expected. Overall assessment: communication objectives mostly met; commercial objectives to be revisited at longer time horizon. Discussion question: In the age of digital advertising with real-time performance data, do brands still need to conduct formal pre-testing of creative work? Or does the ability to run small-scale digital tests and iterate quickly based on real data make formal pre-testing obsolete? The answer: both approaches have a place. Real-time digital performance testing ('A/B testing') works for digital activation advertising where performance is measurable in real time. Pre-testing remains necessary for television and major brand advertising where: production costs are high, media commitments are large, and short-term digital performance metrics do not capture the emotional brand-building effects that are the primary purpose of the advertising. [55–60 min: Summary and Assignment] Today we covered pre-testing methods: focus groups, IDIs, quantitative survey pre-tests, theatre testing, eye tracking, biometrics, and neurometric testing. Post-testing methods: day-after recall, continuous brand tracking, campaign evaluation studies, sales response modelling, and digital analytics. We examined the distinction between explicit and implicit recall effects. We applied post-testing to a Tanishq case study. Assignment: You have a budget of Rs. 5 lakhs to conduct advertising research for a campaign for a Goa-based spice brand. Design a research program that uses Rs. 2 lakhs for pre-testing and Rs. 3 lakhs for post-testing. For each phase, specify the method, the sample size and composition, the key measures, and how the results will be used. One to two pages. Next class — Lecture 37 — we consolidate Unit III with a full Review and Media Plan Exercise. See you then.