Ad Testing: Methods, Metrics and Survey Questions
Pretests, brand lift and tracking, with five metrics and worked creative examples. See how a finding points to a frame, cue or line to change.
Ad testing checks the message, brand connection and reason to act before you spend more on a campaign. It helps you choose a creative direction or a specific edit. A market research plan should distinguish creative interpretation from the campaign outcome you will measure.
Judge the ad against its job: what is remembered, which brand it brings to mind, what it promises and what happens next. Keep those measures apart so an entertaining scene does not hide a missing brand or offer.
Book a demo to compare simulated interpretations of finished ads with the target audience. See our comparisons with human research and keep liking apart from human attention, brand lift and live conversion.
Ad testing at a glance
Study choice | What to agree |
|---|---|
Creative decision | Choose a complete route, a specific edit or the buyer group the creative should serve. |
Exposure | Keep creative versions, placement, device and sound conditions identifiable and comparable. |
Measures | Collect unaided brand and message understanding before detailed diagnostics; keep recall, relevance and next-step expectations separate. |
Action | Connect the response to the frame, cue, wording or destination to change. |
Evidence limit | Modeled interpretation does not establish human attention or live conversion. Campaign lift needs actual treatment/control evidence. |
Ad testing methods
Choose a creative pretest to improve an ad before release, a brand lift study to measure a campaign effect, or tracking to follow the brand over time. Each answers a different question.
Decision | Evidence and returned measure |
|---|---|
Which edit communicates the brand and offer? | Creative pretest with controlled presentation and first-exposure questions. Return understanding, association and reasons by exact edit and audience. |
Did this campaign change a defined brand response? | Brand lift study using an appropriate real treatment/control design. Return the same survey measure, answering bases, absolute lift and uncertainty for the campaign window. |
How is the brand changing over time? | A tracker with consistent audience, wording, fieldwork and measures across waves. Return levels and changes alongside campaign and market context. |
Google's Brand Lift description compares answers from people shown ads with answers from qualifying people not shown them. Its lift-study overview distinguishes brand responses, searches and conversions. An awareness result cannot show extra sales. Choose the design and outcome for the business question.
Ad testing metrics
Metric | What to record |
|---|---|
Unaided brand recall | The share who name the brand without a brand list, among valid answers after the agreed exposure and delay. |
Message understanding | How viewers explain the promise in their own words, with a stated coding rule for correct, partial and mistaken answers. |
Brand association | Whether the intended benefit or occasion is linked to the brand; keep the exact question and answering base. |
Relevance and intent | Responses on the stated scale among the intended buyers. Treat reported intent as an attitude, with purchases measured separately. |
Brand lift | The difference between treatment and control on the same brand measure, with the study design, bases and uncertainty. |
How should the campaign objective shape an ad test?
A brand-building film needs to make the name and its meaning memorable. A product ad needs to explain the offer and give the right buyer a reason to act. An entertaining scene can help one job and distract from another.
Name the choice too. Compare complete ads when choosing a creative direction. For a headline test, keep the image, offer and call to action the same. The result should tell the approver what to change.
What gets remembered?
After first exposure, ask which moments, product and brand people remember. Check whether the remembered scene carries the intended brand or message.
What does the buyer take away?
Ask what the ad means in the viewer’s words. Look for the intended promise, a misunderstanding or a missing part of the offer.
What earns the next step?
Find who sees a reason to act, what stops them and whether the destination delivers what the ad led them to expect.
Agree priorities before seeing results. Keep each measure visible. One average can hide the weakness the edit needs to fix.
Related guides: AI research vs A/B testing.
Match the test to the placement
For Sapien, define the simulated buyer groups and share the creative and placement: feed video, short pre-roll, outdoor panel or carousel. Include device, dimensions, sound and surrounding content. Compare how groups interpret the brand, offer and next step under those stated conditions.
Use the same presentation and question order for each candidate. First ask what it communicated, which brand it showed and what action it invited, without supplying the answers. Then ask which line, cue or detail caused confusion. Compare groups so the team knows which edit serves which audience.
Keep first understanding apart from what becomes clear after a closer look. Naming the brand earlier gives away part of the answer. SurveyMonkey's questionnaire guide explains why recall comes before detailed follow-ups.
View an example study to see the audience, offer and findings before planning your own.
Example: Volvo Trucks’ The Epic Split
In Volvo Trucks’ The Epic Split, Van Damme balances between two reversing trucks. The spectacle happens through the product. A test can ask whether the viewer remembers only a stunt or a demonstration of control linked to Volvo.
The spectacle is the product demonstration.
Role | What to investigate |
|---|---|
Attention | The split creates a moment worth watching. |
Brand | The trucks need to remain part of what gets remembered. |
Promise | The scene should communicate control, rather than an unrelated feat. |
That is an editorial reading of the ad. A test checks what the intended buyer actually takes from it. If the recalled answer stops at “the split between trucks,” probe product and brand meaning separately. If the viewer describes the intended benefit in their own words, preserve the part of the execution doing that work.
Apply that question to a smaller campaign. A joke, celebrity or dramatic opening may get interest while the brand arrives too late. Check how the memorable moment connects with the offer before adding a larger logo or replacing the scene.
Example: Wren’s video opening
Wren is a hypothetical unsweetened iced-tea brand. It is choosing whether a 12-second feed video should open with an afternoon break or the branded bottle. Both edits use the same footage and offer; only their sequence changes.
Wren's 12-second feed video should connect a short afternoon break with the brand and explain the tea's unsweetened taste. The team is considering three four-second clips: a person taking a break with a glass, a labeled Wren bottle pouring tea, and a product end card.
The two edits use the same clips, total duration, soundtrack, text and final invitation, “Explore the flavors.” Only the first two clips change places:
Time | Edit A / Edit B |
|---|---|
0–4 seconds | Edit A: Afternoon break; no legible brand Edit B: Labeled Wren bottle and pour |
4–8 seconds | Edit A: Labeled Wren bottle and pour Edit B: Afternoon break; no legible brand |
8–12 seconds | Edit A: Wren product end card and invitation Edit B: Same end card and invitation |
Should the occasion or branded product come first? Google's video-creative guidance recommends an early brand and focused message. That gives a reason to test B, not proof it will win.
Randomly show each target buyer one complete edit first in the intended setting. Match screen size, sound, captions and surrounding material. If sound-on and sound-off both matter, report them apart. Showing A, explaining it and then showing B does not give an independent first reaction to B.
After the initial exposure, ask what the video communicated, which brand it represented and what the viewer expected after following its invitation. Ask about the product before suggesting “unsweetened.” The purpose is to learn what the execution conveyed on its own. Pew Research Center's question-order guidance explains how an earlier prompted idea can alter a later open answer.
Then replay the same edit to find what needs changing. Ask which moment explained the product or caused confusion and save the timestamp. Correct understanding after replay belongs to that second viewing; it must not overwrite a missing brand or misunderstood promise in the first answer.
If people remember the break but not Wren, test the early product cue. If they recognize the tea but expect sweetness, check the words and demonstration. If the invitation suggests buying but opens a general brand page, fix the destination or expectation.
Swapping the first clips changes both story order and brand timing. Even a better result for B cannot isolate the earlier logo as the cause. It can help choose the complete edit under those conditions. Test the cue alone with a narrower comparison.
Use actual viewing and response records for human exposure or recall claims. Eye movement, skipping and retention need suitable measurements. With Sapien, agree the asset format and viewing conditions assessed. Simulated answers to a storyboard or transcript concern that material; they cannot show what a person noticed during the film.
Give production four things: the tested version, its first interpretation, the scene to keep and the edit to make. Keep that hypothesis with the chosen ad in the live campaign test.
Ad testing questions for brand, message and next step
For the Wren comparison above, collect these first answers after the ad exposure and before naming the brand or explaining the offer. Keep later probes separate so the report shows what the ad communicated by itself.
What you need to learn | Question to adapt |
|---|---|
Brand connection | Which brand, if any, was the ad for? |
Message understanding | What was the ad saying about the drink? |
Buying occasion | In what situation, if any, would this drink fit into your day? |
Next-step expectation | After selecting this ad, what would you expect to find on the next page? |
Ask for the reason behind an answer after recording it. A prompted recognition answer is different from remembering the brand without help; keep those measures separate.
How to act on ad test results
“Improve clarity” leaves creative guessing. Show what was misunderstood, where it began and which part should change.
Scene recalled; brand missing
Bring a distinctive brand cue into the remembered moment. Test that edit while keeping the core idea intact.
Brand recalled; promise unclear
Make the benefit easier to see or say. Cut a competing message instead of adding more explanation.
Promise understood; weak fit
Check the buying occasion and audience. Choose a more useful benefit or the group the offer serves.
Interest earned; next step breaks
Match the call to action and destination to the offer. Check price, product and required action against the expectation.
Ask for the answers and exact ad version behind the recommendation. Keep what works and fix the weak point. Changing headline, offer, image and audience together creates a new ad to assess.
Test the revision in the live campaign
Take the ad and hypothesis into the channel. You might test whether showing the benefit in the opening brings more suitable visitors to the product page. Choose the campaign’s outcome and state the comparison conditions.
Research explains reactions to the creative. A live experiment measures behavior under actual delivery. Report clicks, suitable visits, purchases and costs from campaign data. A favorable research answer is not a conversion result.
Use the result to keep the route and edit the opening, change audience or stop an ad that fails to carry its brand and offer. Decide where further production and media spending is justified.
Brand lift survey and reporting
Wren first needs to choose a 12-second edit. After release, it may need to learn whether the campaign changed awareness or association. Later, it may track that association across campaigns. Each needs a different comparison.
Wren's brand lift brief could name one primary question: “Which of these brands of iced tea have you heard of?” Save the exact brand list and ordering rule, with “None of these” and “Not sure” available. For this measure, selecting Wren is positive; None and Not sure are valid nonpositive answers, while an unanswered item is missing. Use the same question in treatment and control, with the survey timing fixed relative to the campaign. Don't show either group the creative again before that measure. If association with an afternoon break is a separate outcome, retain its own wording, base and result instead of blending it into awareness.
Record the assignment unit, entry rules, campaign dates, creative, placements and other exposure. Ask how the control group is formed, whether the result concerns assigned treatment or verified viewing, and how nonresponse, weighting and other campaigns are handled. People who chose to watch may already differ from people who did not.
Campaign results template
This blank template is for actual campaign evidence. Use one row per campaign, question and planned audience group. Empty results are pending, not zero lift.
campaign_id,creative_set,fieldwork_window,question_id,audience,assignment_unit,treatment_eligible_n,control_eligible_n,treatment_survey_invited_n,control_survey_invited_n,treatment_answers_n,control_answers_n,treatment_survey_response_rate,control_survey_response_rate,treatment_positive_n,control_positive_n,treatment_positive_rate,control_positive_rate,absolute_lift_pp,interval_low_pp,interval_high_pp,interval_method,weighting,exposure_or_assignment_basis
For each arm, positive response rate is positive answers divided by valid answers to that question. Absolute lift in percentage points is 100*(treatment_positive_rate-control_positive_rate) when rates are stored as fractions. Survey response rate is a separate quantity: answers divided by people invited to the survey in that arm. Keep invited counts too if that rate is reported; eligible campaign audience size isn't automatically the survey invitation base.
Show raw answering counts even when the estimated rates are weighted, and state whether the displayed positive count is raw or weighted. Obtain an uncertainty interval from the actual design, accounting for weighting or clustering where applicable. The blank fields above are a result specification, not a reason to apply a simple formula to an incompatible experiment. A small positive point estimate with a wide interval may leave the campaign effect unresolved.
A before/after awareness rise cannot isolate the campaign. Distribution, seasons, competitors and the mix of respondents may change too. Our brand tracking research guide covers that context. A suitable lift experiment answers the narrower campaign question.
The creative report says what to change in the film. The campaign report shows whether the chosen exposure changed the measured outcome. Sapien’s simulated creative comparison can help the first job; it does not create the human exposure and control records needed for lift.
What AI creative testing can tell you
Ask which evidence an AI service returns: visual attention, human interviews or simulated responses. That determines which decision the result can support. To choose a platform for the creative comparison, use our AI market research tools guide to check the response source and research support.
Sapien compares how simulated buyer groups interpret an ad, understand the offer and explain their answers. Use that to choose a route, fix confusion or explore audience fit. Keep each finding with its exact version and question.
A simulated explanation of meaning is different evidence from where a person looked or what they did in a campaign. Choose the evidence for your question. Test product understanding with that measure; test conversion in the actual channel.
See our ad-rating comparison with human research for liking scores and ranks on eight Super Bowl ads, including the largest disagreement. Use the comparison for that task. Ranking agreement cannot show brand recall or campaign lift.
Questions for an ad-testing provider
Before agreeing the study, ask how its presentation, audience and report will help you approve the next ad version.
Compare ad testing platforms by the evidence returned: actual viewers, predicted attention, human forecasts or simulated audiences supported by data. Ask for an output that fits your decision.
- A finding for each ad: brand and message understanding, relevance, objections and the next step it suggests.
- Useful group differences: audiences you can reach, with response totals and measure definitions.
- A specific edit: the frame, cue, wording or destination to change, supported by the answers.
- The response source: human participants or a simulated audience, and the outcomes that need a live campaign.
Agree a plan you can check. It should show how the proposed report will help choose the creative.
Part of the plan | What to agree |
|---|---|
Choice to approve | Whole creative route, a specific element or audience fit; what differs between the versions and what stays fixed. |
Buyer groups | Who needs to understand the ad, the evidence supporting those groups and the comparisons you will use. |
Creative and encounter | Labeled versions, full copy and offer, intended action, placement and presentation conditions; the asset formats the study will assess. |
Questions and measures | First understanding, brand meaning, relevance and reasons, using the questions and measures suited to the choice. |
Returned readout | Findings by version and buyer group, with each question, measure definition, answering base and relevant modeled explanations attached. |
Recommendation | The element to change, the response supporting that change, what to preserve and the next comparison or evidence needed. |
Engagement | Who designs and interprets the study, the included follow-ups, returned report, agreed schedule and complete quote. |
A plan describes the work; the study returns answers and a recommendation. Use the plan to agree the comparison with a client or check that the report will answer the brand team’s decision before naming a winner.
Keep one record per ad, buyer group and question. Fill this blank template with actual findings and mark missing information.
Readout field | Your finding |
|---|---|
Creative version and encounter | — |
Buyer group and eligible base | — |
Exact question | — |
Measure, scale and units | — |
Answering base for this result | — |
Result and evidence source | — |
Explanation and supporting responses | — |
Recommended change and next evidence | — |
With Sapien, compare simulated interpretations of Wren’s openings and ask follow-up questions about the answers. Agree which version or representation will be assessed and what you need to learn before locking the edit.
Book a demo to discuss which opening gives the right buyers the brand, product meaning and next step the campaign needs.
If the open choice is the wording, use our message testing guide.
Creative attribution: Volvo Trucks, The Epic Split, 2013.
Further reading: Kantar's creative testing approach and SurveyMonkey's questionnaire guidance.
Related guides: synthetic research for agency pitches.
