Modeling Testing 101

Modeling & Testing 101
Lead Identification & Qualification
Copyright 2005 Information Resources, Inc. Confidential and proprietary.
> Agenda: Modeling & Testing 101
Todays Agenda
8:30 8:45am 8:45 10:00 10:00 10:45 10:45 12:00pm 12:00 1:00 1:00 1:45 1:45 2:30 2:30 3:15 3:15 3:30 3:30 4:30 Introduction: What is Modeling/Testing Modeling: Drivers Suite of Products Modeling: Store Group Analyses
T
Testing Products: Controlled Store & Matched Market Testing Lunch Testing Products: BehaviorScan Advertising Testing Testing Products: BehaviorScan New Product Testing Panel Models Selling 101 Sales Wizard Exercise
> Introduction: What Is Modeling & Testing?
What Are Modeling & Testing Analyses?
Analyses that isolate the sales impact of specific programs, shelf sets and/or products The results help clients make Go or No Go decisions about using these products/programs/shelf sets in the future Uses statistical techniques
What Are Modeling & Testing Analyses?

Modeling
continued
Testing
Uses disaggregated InfoScan or Panel data + outside data sources Evaluates pre-existing products or programs
Uses disaggregated InfoScan or Panel data + outside data sources Evaluates products or programs that DO NOT exist or not enough data exists to model Therefore, we create the test inmarket
Why Model Or Test?

Why not use panel data or InfoScan data?
Because neither infoscan nor panel data can calculate: The sales impact of price changes TV, radio, print advertising effectiveness FSI, Catalina or ACT media coupon effectiveness The effectiveness of in-store shelf sets or in-store advertising (aside from features and displays) The impact of new products, attributes, packaging or promotions The risk of failure is too high! High dollar investment/downside High personal investment Only 3rd party data support will sell it
Client Business Issues
Your client is under pressure to increase sales by 10% next year
They are considering raising their price 10% to hit their goal but they are unsure if their sales will drop
And oh, by the way, if they dont hit these new goals they will be fired
You pull InfoScan Data Last Year Price This Year Price Last Year Sales This Year Sales
Total US
$1.69
$1.89
200
200
Client Business Issues continued
But upon further digging Year 1 Sales Base Price 200 $1.69 Year 2 200 $1.86 % Change 0% 10%
Promoted Price
Weeks on Promotion
$1.40
8
$1.20
10
-14%
Client Business Issues continued
So
Is
the cost of being wrong high?
Can
the measure sales change do to 10% price increase exist in InfoScan or panel data? your client need precise results for 3rd party support (internally or with retail)?
Does
Would you use TESTING or MODELING to get to the answer?
When to Test vs. Model
As the risk in a decision increases, the need for precision in your answer increases Precision
Risk
Data Already Exists?
Model
Data Does Not Exist?
Test
When to Test vs. Model continued

Client Business Issue Client is thinking about changing glass beer bottles to plastic Should Product-X increase base price by 10%? There is enough price variation within-same-store over the past 104 weeks to run a price elasticity analysis Should Product-Y cut TV ad spending by 50%? Was our Halloween candy promotion as successful this year as last year?
Advanced Analytics Recommended?
Modeling or Testing?
10
Analytic Options
Types
Store Data Regression Models

Isolate the impact of: Trade Promotions Consumer Promotions Price Changes
Store Group Analyses

Isolate the impact of: In-store variables
In-Market Tests
Panel Data Models
Model the sales Test for the sales and consumer impact of NEW: impact of: Products Trade Promotions Product Attributes Consumer Optimal Shelf Promotions Set (Consumer Price Changes View of In-Store variables Category)
11
> Modeling: Drivers Suite of Products
Modeling Options
Types

Mix Drivers/Brand Mix Drivers/Portfolio Price Drivers
In-Market Tests
Panel Data Models
Drivers on Demand
Promo Drivers
12
Modeling Options
Drivers Platform
Price Drivers Promo Drivers Mix Drivers Pricing Merchandising Independent Variables: Portfolio Drivers
TV/Radio Advertising
Couponing Cannib. & Halo
Dependent Variables:
1 SKU
1 SKU
1 Brand or 1 SKU Mix Planner, Mix Drivers CSI
1 Brand or 1 SKU
Mix Planner, Mix Drivers CSI
13
Optional Add-ons:
Price Point / Gap Charts
Price Point / Gap Charts
Statistics Made Simple

Simple regression analysis
To what extent can I predict the change in sales as a function of change in base price?
100
Sales
90 80 70 $0.70 $0.80
Price
$0.90
$1.00
14
Statistics Made Simple continued

Multiple regression analysis
To what extent can I predict the change in sales as a function of change in
Base Price? Promoted Price? In-Store Promotions? Advertising Levels? Coupons? Weather? etc.
How well does the model predict the outcome? R2
For example, an R2 = .95 means that price explains 95% of the variability in sales (if this were a price elasticity model)
15
Modeling Options
Types

In-Market Tests
Panel Data Models
Drivers on Demand
Promo Drivers
16
Mix Drivers/Brand
A multiple regression approach
Weekly storelevel sales Coupon events Trade promotions In-store events Advertising
Bayesian regression models
Incremental Impact of each marketing element
17
Mix Drivers/Brand continued
Quantifies the impact of trade and consumer promotions in two unique ways
1.
By the components that make-up each marketing program

Total Volume: 124,372,360
4.8% 2.3% 4.1%
Special Pack TV Advertising FSIs
Base 66%
Incremental 34%
22.6%
Trade
Base Volume: 82,158,972 Incremental Volume: 42,213,388

18
Quantifies the impact of trade and consumer promotions in two unique ways
2.
By the geographies in which the programs occur
Advertising: DMA-Level
Trade Promotion: RMA-Level

Coupons: Market-Level Store Specific Phenomena: Store-Level
19

Incremental Profit per Dollar Spent
$0.63 $0.57
$0.35
Trade
Advertising
Couponing
20

How does price sensitivity differ in the West vs. the East?
Is Dominics or Jewel more responsive to Feature, Display, TPR?
Which of our brands TV ad campaigns has been most effective in driving incremental sales?
Are Hispanic-targeted trade events more effective than general market trade events in high Hispanic stores?
21
Chose the events with the highest return to optimize ROI
Advertising
Campaign 1 Campaign 2 Feb FSI
Volume per GRP
Couponing
Apr FSI
Volume per Circulation
Event 1
Trade
Event 2
Volume per Week of Support
22
Armed with this level of information, we then optimize marketing spending
Chng in Spending Chng in Vol
0% 5%
Chng in Trade Spending -8% Chng in Coupon Spending Chng in Ad Spending -2% 10%
23
Questions It Can Answer: What mix of advertising trade promotion, couponing, etc. will maximize an individual brands sales & marketing ROI? Which of our brands TV ad campaigns has been most effective in driving incremental sales? How effective are my Hispanic marketing efforts? Should we do more or fewer coupons? Which face values have worked best?
Pricing: Average: $110,000 Timing: Results: 8 12 weeks from signature
24
Modeling Options
Types

In-Market Tests
Panel Data Models
Drivers on Demand
Promo Drivers
25
Mix Drivers/Portfolio
Estimates the impact of a brands marketing activities on:
its own sales the sales of other brands in a companies portfolio and thus the entire portfolio accounting for all across -brand cannibalization (and halo) effects
26
Mix Drivers/Portfolio continued

% Volume Profit Impact Impact ($MM)
What is the net impact on the portfolio of the Stars cereal brands $5 million TV ad spend? Total Portfolio +10% +$5
+8%
+$4
+4%
+$2
$5mil - 2%
-$1 0
27
0
Mix Drivers/Portfolio continued
Questions It Can Answer: How much of our marketing budget is causing consumers to switch brands vs. generating incremental volume for the company? How can marketing spending be reallocated across brands and across marketing vehicles to optimize total portfolio profitability and marketing ROI? Which marketing activities generate the most incremental volume per marketing dollar spent, for each brand and for the portfolio overall? Which brands experience the most marketing-driven cannibalization, i.e., Brand X loses a lot of sales when Brand Y is on promotion? Which brands' advertising has positive "halo" effects on other brands' volume? Pricing Average: $500,000 for 10 brands Timing Results: 12 16 weeks from signature
28
Modeling Options
Types

In-Market Tests
Panel Data Models
Drivers on Demand
Promo Drivers
29
Price Drivers & Promo Drivers
Addresses 5 issues:
Impact of increasing/decreasing own base price Impact on a competitor of increasing/decreasing own price
Additional impact, if any, of crossing specific price points

Additional impacts, if any, of crossing price gaps Impact of increasing/decreasing promoted price by promotion type
30
Price Drivers & Promo Drivers continued

Elasticity = -1.1 A base price increase of 10% on SKU X would result in a volume decline of 9.1% in Grocery
30% 20% 10% 0% -10% -20%

-20% -15% -10% -5% 5.2% 0% 0.0% 5% -4.7% 10% 15% 20%
Grocery 24.9% 17.6% 11.1%
-9.1% -13.0% -16.6%

31

What is the highest price I can charge without crossing a price threshold and losing a significant amount of volume?
Percent ACV at Each Price Point
30.5 21.5 17.2 17.9
Avoid crossing the $1.39 price threshold
10.7
12.1
Large volume loss above elasticity effect
$0.99 $1.09 $1.19 $1.29 $1.39 $1.49 Sales Rate 15.1

14 14 14 14 7.9
32

% vol decrease expected from a 10% price increase in stores that do not cross the $1.39 Price Threshold
An additional -13.4% volume decrease can be expected in those stores that cross the $1.39 Price Threshold
-9.1%
-22.5%
33
Findings are coupled with client financials to calculate ROI

Impact of 10% Base Price Increase SKU X
-9.1%
Volume
-4.8%
Dollars
Not worth it!
-2.0%
Profit
34
SKU Y will experience 0.5% increase in volume sales for every 1% increase in the everyday price of SKU X
Elasticity
Cross Elasticity
Joint Distribution*
SKU X
-1.1 0.5
99%
SKU Y
35

1600%
Percent Increase
TPR Only 1200% 800% 400% 0% 0% 10% 20% 30% Percent Price Cut 40% 50% Feature Only Display Only Feature & Display
36

Scenario 1 Maintain SKU X Price Decrease SKU Y Price Scenario 2 Increase SKU X Price Decrease SKU Y Price Scenario 3 Increase SKU X Price Maintain SKU Y Price
5.7% 2.2% 0.2% -0.7% -0.3% -0.1% -2.5% Volume Dollars Profit -6.5%
37
1.2%
Price Drivers
Questions It Can Answer How will sales react to base/promotion price changes? How sensitive is my product to changes in price or price gaps of a competitors product? How will sales react if I raise the price on my small size but not my large size? Will small size buyers trade up (increasing total volume) or will sales decline? How will Hispanics (or other demographic groups) likely respond to a base price increase/decrease? Is there a critical price threshold for my product; that is, will sales decline dramatically if my price exceeds $2.99? Pricing $20,000 $80,000 Timing Results: 4 8 weeks from signature
38
Promo Drivers
Questions It Can Answer All questions posed for Price Drivers, plus What trade promotion vehicle performs best in Dominicks stores in the Midwest? Which account does it perform best in? Did my in-store display reach families with no children as well as those with children? How much profit did my feature generate for both my brand and the retailer? Does my 64-oz size product perform best with a feature, display, TPR, or feature and display, or frequent shopper feature? How can I built a promotion planning tool that my sales force can use to spend trade funds more profitably? Pricing Average: $60,000 Timing Results: 4 8 weeks from signature
39
Modeling Options
Types

In-Market Tests
Panel Data Models
Drivers on Demand
Promo Drivers
40
Drivers on Demand
A web-based reporting application that provides a model-informed explanation of sales changes vs. last period Due Tos in easy-tointerpret, click-to-drill down bar charts
Price & Promotion Elasticities Created for the top 95% of all SKUs in the category Can be exported for use in price and promotion simulations and planning tools Website Standard interface enabling access to reports and range of key drilldown options Additional customization available
41
DoD Demo: Sales down 32% in the East
42
DoD Demo: Brand sales down 27% in Pittsburgh
43
DoD Demo: 2L size down 46% in Pittsburgh
44
DoD Demo: Main problem on 2L size in Pittsburgh

Avg % discount was 36% last year, now its just 23%
45
Drivers on Demand continued

Questions It Can Answer How can we reduce the amount of time we spend analyzing the InfoScan tracking data and get to the heart of the matter: what variables are driving what percent of our sales change versus year ago? How can we take what we know about price elasticity, promotion lifts, etc. and help our entire sales & marketing organization apply it to their interpretation of our sales trends? How can we execute better and faster response to competitive price and promotion changes, as well as understand the impact of our own decisions? Pricing $150,000 $800,000, 1 year subscription Timing Lead time: 2 3 months from signature Data updated 2 5 days after clients InfoScan database is updated
46
> Lets Help These Clients: Pricing & Merchandising
Client Business Situation
Your client faces increased manufacturing costs for their paper towel product and would like to take a 10% price increase. The brands price has been increased by this percentage or more in the past, but they are now afraid of crossing the $1.99 price threshold. They want to know what the volume impact of crossing that threshold will be. What type of analysis would you suggest and why? What if they inform you that they have very good retailer relationships and no retailers have ever priced the product above $1.79? What would you then suggest?
Your client currently employs line pricing across their portfolio of breakfast cereals. They plan to begin adjusting price independently for each brand and will custom-implement this retailer-by-retailer over several years. They would like to track the impact of this at brand and SKU levels by retailer as it rolls out. What type of analysis would you suggest and why? What if they also plan to adjust advertising spending at the market level as the price changes take effect? How would you suggest reading the combined effects of the advertising and price changes?
47
A manufacturer of home cleaning products is re-evaluating its pricing and promotion strategy for its all-purpose cleaner brand in response to sales declines over the last year, presumably due to a price cut by a key competitor. Should they cut their price in defense? If so, by how much? Or would it be better to maintain the current base price and do more or deeper promoted price discounts? What types of trade promotions are most effective? Should their strategy differ for their extra large size, which does not have direct competition? Should their strategy differ by retail account or by region of the country? What type of analysis would you suggest and why? What if the client wanted to also re-evaluate its advertising? What would you then suggest?
48
> Modeling: Store Group Analyses
Modeling Options
Types

Store Group Profiler Ethnic Workbench
In-Market Tests
Panel Data Models
49
What Is Store Group Profiler (store group analysis)?
Group stores by those with and those without an in-store condition
Stores With Display
Stores Without Display
IN A NUTSHELL: Group stores by a certain condition and compare sales between the groups of stores.
50
Then Analyze The Difference In Sales Rate

Brand-A Sales Rate
Displays = 36% increase in sales @ 95% confidence level
1.64 1.21
Stores with Displays
Stores without Displays
51
Often Uses At Least 1 Of 3 Sources Of Information
InfoScan disaggregated store data Sales rate Display vs. no display

Audit data from Mosaic or Client Data Store addresses with aisle floor sticker Shelf stocked vertically vs. horizontally Spectra data for demographics 50% or more Hispanic 50% or more Asian
52
Store Group Profiler

What store location provides the highest sales rate for my breakfast bar brand?
Sales Rate by Stocking Location 1.64
Not an effective stocking location Snack aisle has highest Sales Rate
1.29 1.01
Cereal
% ACV:
Snack Bar
24%
Candy
12%
64%
53
Another Approach: Top/Bottom Stores

What variables differ between my brands top performing stores vs. bottom performing stores?
% of Stores with Pharmacy
98%
Almost all Top stores have pharmacies, half of which are within 10 ft. of category
30%
Percent of Stores with Pharmacy within 10 ft. of HBA Category
54% 22%
Top
Bottom
Top
Bottom
54

Questions It Can Answer Which of the four most common stocking locations results in the highest sales for my brand? Does my brand have an opportunity to grow sales in stores with predominantly Hispanic shoppers, African-American shoppers, or another demographic group? How am I priced relative to my competition? How do my sales respond to an increased number of flavors in my product line? What are the key differences between the top- and bottomperforming stores that carry my product? Pricing $15,000 $40,000 Timing Results: 5 7 weeks from signature
55
Modeling Options
Types

Store Group Profiler Ethnic Workbench
In-Market Tests
Panel Data Models
56
Ethnic Workbench
Tracks sales in Hispanic and African American markets and RMAs Uses Spectra to groups markets/RMAs based on % of Ethnic Group surrounding the store
Brand X Dollar Sales Percent Change, Grocery Outlet
Total Mrkt Hispanic Mrkt 11.2% -1.2% Total US 11.4%
44.6% 26.9%
50.6%
San Antonio
Retailer A
57
Ethnic Workbench continued
Basic Service: Hispanic markets only (>=50%) Set selection of markets and RMAs APMD maintains projections with 2 years of back data Custom Service: Hispanic and African American Client determines the percent ethnicity Client can define custom groups dependent on sufficient sample Local ATS creates and maintains projections
58

Markets Total U.S Los Angeles San Diego Phoenix/Tucson West Texas/New Mexico San Antonio Dallas Houston Outlets FDMx, Food, Drug FDMx, Food, Drug FDMx, Food, Drug FDMx, Food, Drug FDMx, Food, Drug FDMx, Food, Drug FDMx, Food, Drug FDMx, Food, Drug
Miami
Chicago New York
FDMx, Food, Drug

FDMx, Food, Drug FDMx, Food, Drug
MassX is not releasable as an individually reported outlet
59

Questions It Can Answer Which brands are over- versus under-developed, or growing versus declining, in ethnic markets and vs. the general market? How effective are our ethnic-targeted marketing efforts in driving incremental sales in the defined "ethnic" stores within specific cities and retailers? Are "ethnic" stores carrying the optimal mix of products in our category? Are we getting the desired level of distribution on our ethnic-targeted SKUs in ethnic stores? Pricing $40,000 $96,000, 1 year basic service Timing Results: 8 weeks from signature
60
> Lets Help These Clients: Ethnic Marketing
Your client consistently allocates a large percent of their marketing budget targeted toward Hispanic consumers and would like to track how effective their ethnic-targeted marketing efforts are in driving incremental sales.

What type of analysis would you suggest and why? What if your client only performed an ethnic-targeted event once and only wanted to see the effect of that one event, what would you then suggest?
Your client is interested in developing ethnically targeted marketing, but would first like to know, in general, if there is an opportunity to grow sales in stores with predominately Hispanic or African American shoppers.
What type of analysis would you suggest and why?
61
> Lets Help These Clients: Ethnic Marketing
Your client manufactures flour, and has introduced bilingual displays in stores serving Hispanic communities. Sales of their brand and the category have noticeably improved in markets where they received display support. Now they would like to collect some more specific data to build a sales story for their retailer customers. They want to know what kind of sales increase they can tell a retailer to expect from these displays in Hispanic-concentrated areas.

What type of analysis would you suggest and why? In a subsequent discussion, the client expresses interest in knowing the true ROI they receive from their investment in this display support. To address this issue, what type of analysis would you suggest and why?
62
> Testing Products
Testing Options
Types
In-Market Tests
Panel Data Models
Controlled Store Test

Matched Market/DMA Test
BehaviorScan
63
> Testing Products
Basic Testing Concept

Statistically matched groups of test and control markets or stores
Execute & monitor conditions in test & control markets or stores

Perform statistical analysis of results to measure sales impact
Sales Pre-Test Test Stores/ Market/DMAs Execute Test Post-Test
Net Result
+3
Control Stores/ Market/DMAs

64
> Testing Products

Match
Match: Identifies groups of stores that are similar in terms of sales rates and sales trends. Gives you the test group and the control group
1200 1000 800 600 400 200 0 6/1/2000 9/1/2000 11/1/2000 1/1/2001 3/1/2001 5/1/2001
Test Cell
Control Cell
65
> Testing Products

Match, Analysis of Covariance (ANCOVA), Confidence Level
ANCOVA: Adjusts for any brand or competitive cross-cell differences aside from the test variable
Test Cell Unadjusted Vol/1000 HHs
Covariate Adjustments Potential Adjustments:
Test Cell Adjusted Vol/1000 HHs
Test Effect
Control Cell Unadjusted Vol/1000 HHs
Price Trade Etc.
Net Difference Confidence Level
Control Cell Adjusted Vol/1000 HHs
Confidence Levels: A 99.7% confidence level means that we are 99.7% confident that differences between the test and the control group are significant
66
> Testing Products
Testing Options
Types
In-Market Tests
Panel Data Models
Controlled Store Test Matched Market/DMA Test BehaviorScan
67
> Testing Products

Statistically matched groups of test and control stores Execute & monitor conditions in test & control stores Perform statistical analysis of results to measure sales impact
Sales Pre-Test Test Stores Execute Test Post-Test
Net Result
+3
Control Stores
68
> Testing Products

Packaging
Sales of the shrink-wrapped package were 39% below sales of the box package
Avg. Weekly Volume/$MM ACV
PIZZA
-39%
69
> Testing Products

Permanent Display

16% of all market households bought the test product Category dollar sales (driven by this leading vendor) rose 5% following the installation of this major display
% of Market HHs Buying Test Product
16%
% Change in Category Sales
5%
70
> Testing Products

Pricing
The 25 and 35 price points significantly outperform the 45 control cell and both meet their predetermined hurdle rates
Net Effect on Sales
194.2%
135%
112.9%
57%
Hurdle Rates
38.2%
$0.25 vs. $0.45
$0.35 vs. $0.45 Price Points
$0.25 vs. $0.35

71
> Testing Products
Controlled Store Testing

Questions It Can Answer If Brand X's price is increased by 10%, how will sales of the brand and key competitors be affected? If the category shelf set were arranged differently, would category sales go up? Will in-store sampling or couponing pay out for this brand? Will this new package design help (or hurt) the product's sales? Do "customized" or "themed" display racks produce more incremental sales than cheaper "generic" displays? Pricing $50,000 $250,000 Timing Lead time: 6 8 weeks Results: 6 8 weeks from end of test
72
> Testing Products
Testing Options
Types
In-Market Tests
Panel Data Models

BehaviorScan
73
> Testing Products
Matched Market / DMA Testing
Just like a Controlled Store Test except it measures events that can not be separated at the store level
Television
Billboard
Test
Control Test
Radio
Couponing
Control
Subway
Bus
74
> Testing Products
Matched Market / DMA Testing

Advertising Testing
Percent Change in Volume Sales
70%
72%
No significant difference
19% 2% $3 MM TV Ad vs. No Ad 9% $8 MM TV Ad vs. No Ad
9% $11 MM TV Ad vs. No Ad Total Brand
Line Extension*
*Line Extension sales represent 5% of Total Brand Sales

75
> Testing Products
Matched Market
FSI Coupons
Net Sales Gain, Use on Any vs. Restricted Coupons
36%
18% 12% 3% Total Brand Form A Form B AO Forms
76
> Testing Products
Matched Market Testing

Questions It Can Answer If advertising (TV, radio, print, outdoor, transit) were introduced, how much would brand sales increase? Will this new TV ad copy help (or hurt) the products sales? How much will sales decline if advertising is discontinued? Will FSI couponing pay out for this brand? How will brand and category sales respond to a 50% increase in advertising? Will a consolidated multi-brand FSI achieve better payout than individual FSIs for each brand? Pricing $35,000 - $60,000 Timing Lead time: 2+ weeks Results: 6 8 weeks from end of test
77
Your Salty Snacks manufacturer client is considering a merchandising overhaul new fixtures and endcap displays with POS, and reducing their heavy TPR activity by 50%. The new fixtures will be quite expensive to produce and install in retail stores. What type of analysis would you suggest and why? What if the client has already rolled out the new program in a number of stores within an RMA? What would you then suggest? Your client manufactures three brands: a laundry detergent, a fabric softener and a pre-soak treatment. They would like to evaluate the ROI they receive for their features, displays, TPRs, coupons and television advertising. What type of analysis would you suggest and why? What if their three brands were all laundry detergents? What would you then suggest? What information could the client now receive that they couldnt under the laundry/softener/pre-soak scenario?
78
Throughout the past year, your dairy client has been securing feature ads for their gourmet cheeses in four markets: Los Angeles, New York, Chicago and Dallas. InfoScan data is unclear as to whether these features have driven true growth for their brand. They are considering expanding this program to more markets but are unsure whether the investment truly pays out. They need the answer as quickly as possible as critical planning meetings are coming up next month. They have a budget of $40,000 for this research. What type of analysis would you suggest and why? What if the client also wants to evaluate the TPRs they have been running nationally, and compare the return on these TPRs to the return on the feature ads in the four markets? What would you then suggest?
79
> Testing Products
Testing Options
Types
In-Market Tests
Panel Data Models

BehaviorScan
80
> Testing Products
BehaviorScan - Markets
Cedar Rapids, IA Grand Junction, CO
Eau Claire, WI
Pittsfield, MA
Targetable TV No Targetable TV Midland, TX
Multi-Outlet Panel: Cedar Rapids & Eau Claire

81
> Testing Products
BehaviorScan continued
Permanent, experienced staff that:
Obtains retailer authorization, shelf placement and SKU deletions Warehouses the test product Delivers to stores Executes promotions, perform shelf audits, etc.
82
> Testing Products
The
Only Split Cable Advertising Testing System In The Country Invisibly deliver different TV commercials To distinct household groups in the same city
Broadcast Ad
IRI Cut-In Ad
83
> Testing Products
Checkerboard System
Markets have 30+ nodes, grouped into three demographically balanced cells Covers all cable TV channels in that market and all TV sets in the household Large sample for survey research No panelist workload
C A
A B
C B B
B A
84
> Testing Products
Dual-Mode Panel tracks purchasing in more outlets
Panelists continue to show their card at participating stores Also use a handheld scanner at home for purchases at non-participating stores Dual approach minimizes panelist burden
Available in Eau Claire & Cedar Rapids
Captures
85
> Testing Products
Coverage

Weekly store data from nearly all grocery, drug, and mass stores Weekly collection of feature ads and displays Data from other sources can also be incorporated
Participating in each BScan market are:

7
to 12 FOOD stores
1
3
to 14 DRUG stores
to 5 MASS stores*
*Mass stores are not under contract but can be approached on a test-by-test basis
86
> Testing Products
Advertising Testing
National or DMA Condition
Local Cable System

BehaviorScan Market Studio Cell A National Ad Cell B + Cell C No Ad Non-Cable Subscribers
National Ad
Control Cell
Test Cell
87
> Testing Products
Advertising Testing
Adjusted % Net Difference - This quantifies the lift in sales from the test treatment, while taking into account outside variables that could exaggerate the true effect. Significance Level - Represents the confidence that the impact was not just due to random variation in data. Industry standards consider 80% SL to be statistically significant. So, in this example we would say that the effect is highly significant. 99 times out of 100 we would expect to see an impact from the test treatment. Confidence Interval - The confidence interval quantifies the range in the test effect if the experiment were repeated, given the variance in the data. This is impacted by the strength of the significance level. In this example, 80 times out of 100, we would expect the results to be between 5% and 15%, with our best estimate being 10%. The smaller the brand and the smaller the analyzed geography, the wider the confidence interval.
Adjusted % Net Difference* in Volume Sales Rate 10%
Significance Level 99%
80 times out of 100, the impact from the Radio Promotion will be between +5% and +15%. 80% Confidence Interval
Upper Bound Best Estimate Lower Bound
15%
10%
5%
88
> Testing Products
Advertising Testing
Lower media costs in small markets offset higher research costs BehaviorScan Markets Media Costs* Pittsfield (Albany), MA Midland, TX $90,000 Matched Market Atlanta, GA Pittsburg, PA $330,000
Research Costs**
Total Costs Savings
$200,000
$290,000 $90,000
$50,000
$380,000
* Media Cost: Assumes $10MM National Theoretical Spot Buy ** Research Cost for B-Scan: 24-week contract (2 markets) and analytics (match, ancova, Psum, Brand Switching) ** Research Cost for MMT: IRI analytics (match & ancova) only, 3 outlet.
89
> Testing Products
BehaviorScan vs. Matched Market / Matched DMA

BehaviorScan Split-Cable Test and Control Cells in the Same Market Market: Eau Claire Market: Phoenix Test Market: Dallas Matched Markets/DMAs Test and Control Cells in Different Markets
Test
Control
Control
90
> Testing Products
BehaviorScan vs. Matched Market / Matched DMA

BehaviorScan Ad Testing
Matched Market Testing
> 50% of Sales in Grocery (multioutlet panel available in Eau Claire and Cedar Rapids)
Need to read Total FDMx
Targetable advertising at household level (e.g., direct mail literature, coupons, newspaper ads) Total FDM and cut-over part of a national plan
Not targetable advertising (e.g., radio, billboards) BScan sample size is insufficient to read a low-incidence test When client planned a roll n read, but later had difficulty separating the noise from the test effect
91
> Testing Products
Advertising Testing
Questions It Can Answer What would happen to sales if we had to turn off all TV advertising for a year? Would doubling our TV ad spending generate a pay-out level of incremental sales? Is this new ad campaign really better than our current campaign? Would advertising in the "off" season be worthwhile? Pricing $150,000 - $250,000 Timing Lead time: 2 4 weeks Test length: average is 24 52 weeks Results: 6 weeks from end of test
92
Lets Help These Clients: Advertising Issues
Your client manufactures 5 different cookie brands. They run advertising on three of these brands and would like to determine how much incremental volume is generated by advertising for each brand, and how much of this incremental volume is cannibalized from their other brands. What type of analysis would you suggest and why? What if they also wanted to know how much incremental volume they could expect if they began to air advertising on the two unadvertised brands? What would you then suggest?
Your client is currently airing TV and radio advertising for their brand in 18 markets around the U.S., and due to budget cuts is considering cutting the TV and radio advertising by 50% in 6 of these markets. They would like to know what the volume impact of this change will be. What type of analysis would you suggest and why? What if they were only planning to cut the TV advertising by 10% in all 18 markets? (No change to radio.) What would you then suggest?
93
Lets Help These Clients: Advertising Issues
Faced with potential advertising budget cuts, a client needed to: Prove the importance of advertising overall Identify ways to increase ROI by optimizing the dollars allocated to each media vehicle (TV advertising, print, & outdoor) The client expects the cut to be about 10%.

What type of analysis would you suggest and why? What if the client tells you that they expect the cut to be 50%?
94
> Testing Products
BehaviorScan
New Product Testing
Since we have:

The power to sell in, warehouse and stock the products

Weekly store data Weekly feature ads and displays
Weekly Panel data

Advertising Data
We can
95
> Testing Products
New Product Testing
Estimate National Sales in 12 weeks
Year One Estimated National Unit Sales (MM)
13.59 MM
35% of goal
4.73 MM
Year One Forecast
Year One Goal

96
> Testing Products
New Product Testing
Cumulative Trial vs. Objectives

New Product A - % of Households Trying
6.0% 5.0% 4.0% 3.0% 2.0% 1.0% 0.0%

Cum Adjusted Trial Goal 9/30/01 0.1% 1.0% 10/28/01 0.4% 1.9% 11/25/01 0.8% 2.9% 12/23/01 1.1% 4.0% 1/20/02 1.3% 4.8% 2/17/02 1.6% 5.3%
97
> Testing Products
New Product Testing
Since we have:
Volume Decomposition
23.0% Merch Base Volume 16.9% Incremental Volume 83.1% 22.4% TV 5.1% In-pack 23.6% Consumer 9.0% Print
98
> Testing Products
New Product Testing
Simulate the sales impact of different marketing plans
Year One Estimated National Unit Sales (MM) 61% of forecas t
4.73 MM
2.90 MM
Low Spend Scenario
Year One Forecast

99
> Testing Products
New Product Testing
Questions It Can Answer For major innovations or brand restages, what is the national yearone sales potential? How can we improve our introductory marketing plan? Are we reaching the right consumers with the right message? To what extent will this new product generate incremental sales for our company or the category? Pricing $300,000 $500,000 Timing Lead time: minimum of 6 weeks Test length: 16 52 weeks Results: 6 8 weeks from end of test
100
> Testing Products
When Should BehaviorScan Be Used In Addition To A Simulated Test Market?

When An STMs Is Enough

When Bscan Should Be Incorporated

Established category Cannibalization not important
New or multiple categories Cannibalization important
Typical marketing plan

Low investment Low risk Confidentiality critical
Unique marketing plan

High investment High risk Some exposure acceptable
Need results in 6-8 weeks
Need results in 12-24 weeks
101
> Testing Products
New Product Testing
Product Requirements (cases)1 x Profit per Case Product Profit2 Slotting Testing Contract Fee3 Product Stocking/Handling Promotion Execution Media Production Media Execution3 Media Cut-Ins
10% ACV Regional Test

108,000 $4.00 $432,000 $1,000,000 $0 $0 $50,000 $500,000 $500,000 $0
Bscan 828 $4.00 $3,312 $0 $180,000 $50,000 $10,000 $100,000 $100,000 $50,000
Analytics
In-Market Test Costs Net Cost for Testing Options
$10,000
$2,060,000 $1,628,000
$90,000
$580,000 $576,688
1: Assume 24 week test, 12 units per week, 3000 stores for 10% ACV test, 23 stores for Bscan 2: Assumes all product sells through with no left over/distressed inventory at end of test: 3: Assume 2 markets are required in BScan for 24 weeks; 4: Assume 500 GRP prime-time plan
102
> Panel Models
Panel Options
Types
In-Market Tests
Panel Data Models

Market Structure Attribute Drivers
103
> Panel Models

Logit Models
Model the consumers choice to make predictions about product substitutability or inherent value of product attributes
104
> Panel Models
Panel Options
Types
In-Market Tests
Panel Data Models

105
> Panel Models
Market Structure
Gives you insight into how consumers view a category. Specifically, it identifies the consumer decision tree
Are any of our brands directly competing with each other such that one could be scaled back?
No differentiation; cut SKUs
Are there small products that meet a distinct consumer need, and need to stay on the shelf?
Low Sodium
Where are the growing segments that I should try to get in?
Low Carb!
106
> Panel Models
Market Structure continued

Hot Dogs
Better For You
Branded
Value
Client Brands
Competitor A & Private Label
Client Brands
Competitor Brand
Competitor Brands
Action Taken: For more than 20 years, our poultry client had been pricing their chicken hot dogs based on Ball Park Beef Franks. Combined with a price elasticity study, they were able to raise their price and increase profitability. Able to get retailers to delist Competitor A b/c it cannibalized Private Label sales
Result: Sale up 10%, Profits up 25%

107
> Panel Models
Market Structure continued

Questions It Can Answer What brands are considered my closest competitors? Should I consider products in category A to be a competitive threat to my brand in category B? How differentiated are my brands? Is there any new or under-served segment in which I'm not competing, but could/should be? How important is variety in this category, i.e., how many unique segments are there? To maximize category traffic in any given week, which brands and product segments should be promoted together vs. separately? Pricing $65,000 $155,000 Timing Results: 12 16 weeks from signature
108
> Panel Models
Panel Options
Types
In-Market Tests
Panel Data Models

109
> Panel Models
Attribute Drivers
You Know Whats Driving Incremental Sales But What About Base Sales? % of Sales due to . . Base/Everyday Volume driven Brand name by consumer benefits Package type Package size Base price Product form or shape Product color, flavor, or scent Nutritional content: fat, salt, vitamins, soy, etc. Product ingredients: aloe, cocoa butter Product benefits: strengthening, whitening, Incremental volume cleansing due to marketing
110
> Panel Models
Attribute Drivers continued
Learn which Product Attributes are Most Important to Consumers
Type 36% Base Price 8% Form 12% Size 17% Brand 27%
300
With Sunblock Oil-Free
208
100
Original
Type is driving consumers purchase decisions in the category
With Sunblock Preferred 3-to-1 over Original

111
> Panel Models

Application: Evaluate Net Effect on Brand & Category
Implementing the projects recommendations will increase the companys retail sales by $8.1 MM and the category by $2.0 MM.
Projected Impact of Sun Block Intro

Total Lotion Category Total Company New Sun Block SKU Our-Brand-A (regular) Our-Brand-B (sensitive) Our-Brand-C (scented)
-$1.9 -$1.3 -$1.2 $2.0 $8.1 $12.5 (New)
(Cannibalization)
112
> Panel Models

Questions It Can Answer What product attributes are most important to consumers in this category? How unique is our brand to consumers? Which brands in the category are most valued by consumers? Were considering introducing new SKUs in the coming year; what flavors, sizes, forms, scents, etc. are most desirable and how much will consumers pay for them? What product attributes are important to light/non-loyal buyers? I want to introduce a new flavor; how will a new SKU impact my share? Pricing $70,000 $100,000 for category analysis $30,000 $100,000 for simulations Timing Results: 12 14 weeks from signature
113
> Panel Models
Attribute Drivers vs. Market Structure

Use Market Structure to Identify competitive sets Use Attribute Drivers to Run SKU addition/deletion simulations based on these competitive sets
Energy Bar-A
-2%
Cat AO
Granola BarX competes with Energy A/X Bar X
A new low-carb bar would steal 2 share points from Energy BarA
Identify key category attributes

Cat Reg Low
Quantify the importance of these attributes

Consume s will pay $0.40 more for low-fat
114
Low-Fat vs. Reg Fat is the 1st break
Full Fat Low Fat
> Lets Help These Clients: New Product Issues
Your client manufactures hard candies, and this year the Watermelon flavor is hot. They had been planning to introduce a Mango flavor to mimic their main competitor, but are now considering launching Watermelon instead (following in the footsteps of several small manufacturers) and want to know which flavor would generate more sales for them. What type of analysis would you suggest and why? What if the Watermelon werent yet in the marketplace? What would you then suggest? Your client is a soft drink manufacturer who is thinking of launching a line of fruit juices. They have never competed in the Aseptic Juices category and need to get up to speed on the category basics: what are the key segments and which are complementary vs. competitive? What type of analysis would you suggest and why? What if they also wanted to understand the ways in which the Aseptic Juices category interacts with the soft drink category? What would you then suggest?
115
> Lets Help These Clients: New Product Issues
Your client is an ice cream manufacturer that plans to introduce a new line of extra-smooth ice cream with a superior mouth feel. To manufacture this product, they will need to make a $30MM investment in retooling their plant. They want to know what kind of volume they can expect in Year One of their launch to recoup this cost . What type of analysis would you suggest and why? What if they also arent sure whether to support the launch with $25MM or $50MM of national advertising? What would you then suggest?
116
> Selling 101
Who buys our products?

Marketing Responsibilities
New Product Line Extensions
Sales Responsibilities
Current Products Trade Promotions
Shelf Placement/Space
Product Mix
New Products How do I sell it in to retailers?
Price
Advertising
Coupons
117
> Selling 101
How do you sell?
N E A D S
What are you using NOW to solve the problem? What do you ENJOY about the solution What do you ALTER about the solution Who is the ultimate DECISION MAKER?
Then & only then do you talk about the SOLUTION?
118
> Sales Wizard Exercise
The Big 10 AIG Services

Attribute Drivers BehaviorScan Controlled Store Testing Drivers on Demand Ethnic Workbench
Matched Market Testing Mix Drivers/Brand Market Structure Price / Promo Drivers Store Group Profiler*
Services highlighted in red are those where CSG is typically more closely involved with AIG in selling/execution/delivery
119
Review of Solution Matrixes
>Solutions Matrix
IRI Solutions Matrix for Ethnic Analysis

Client Business Issue Store Ethnic Group Workbench Analysis
One-time assessment of ethnic promotion

Ongoing tracking of ethnic sales performance
= good solution, = best solution
121
> Solutions Matrix
IRI Solutions Matrix For Advertising

Client Business Issue Need to compare impact of advertising to that of other promotion vehicles Need to evaluate advertising that has already aired Ad spend/weight is changing by more than +/-25% Ad impact for small/regional brands needed Have an existing national media plan, will go dark in some markets to test Usually buy spot markets, will buy additional weight/markets to test Non-TV media figure heavily in the mix (radio, print, outdoor) BScan Mix Drivers Matched Market
122
> Solutions Matrix
IRI Solutions Matrix For New Products

Client Business Issue Define the category boundaries and segmentation Identify segments Need to understand how consumers shop the category Quantify volume potential of a close-in line extension Quantify volume potential of a highly innovative, highrisk new product Rank consumers flavor preferences (all else being equal) Market Struct. Attrib. Drivers BScan Test
Evaluate the impact of different advertising support levels for the launch
Identify SKUs to eliminate to make room for new items
123
> Solutions Matrix
IRI Solutions Matrix For Pricing

Client Business Issue
Store Group (NonAudit)
CST
DoD
Mix Driver/ Brand
Mix Driver/ Portfolio
Price/ Promo Drivers
Quantify impact of price changes less than +/-25%, where historical price variability exists Compare pricing effects to merchandising/advertising effects Evaluate unprecedented pricing conditions or changes Track price change impacts on an ongoing basis Determine price elasticities Learn what price points and gaps exist in the marketplace

124
> Solutions Matrix
IRI Solutions Matrix For Merchandising

Client Business Issue
Track impact of merchandising events on an ongoing basis Evaluate ROI of features/displays/TPRs Compare merchandising ROIs to advertising ROIs Evaluate unprecedented in-store promotion conditions Evaluate ROI of coupons or multiple, complex events
Audit Store Group
CST
DOD

Mix Drivers/ Brand
Mix Drivers/ Portfolio
Price/ Promo Drivers
Measure impact of specific nondisplay, in-store elements

Determine halo/cannibalization effects within a portfolio of brands in the same category

125
> Solutions Matrix
IRI Solutions Matrix For Assortment

Client Business Issue Need information that is specific to each retailer Have a hypothesis about best SKU mix that is in some stores now, just want to prove it Need national information on which product features are most important Want to evaluate shelf layout scenarios in addition to SKU mix (With Audit)
Attrib Drivers
126
Appendix Media 101
> Media 101
Ad*Views: Critical to Understanding Advertising
Provides advertising data on existing or potential clients Standard pull for models National/DMA-level brand detail $$ and HH GRPs by week/creative/daypart/ad length Data Available 2 weeks after execution of GRPs and 6-8 weeks for spending Input to Mix Drivers & New Product Benchmarks, & used to verify Match Market delivery
Data Request form & Category access is on Sharepoint AIG/Media Services/AdViews

Standard turnaround is 5 days Contact Holly Wasserman 312-474-8618 for Ad*views requests/questions etc (media flowchart will facilitate questions )
133
> Media 101
Key Terms - Ratings
Rating: % of Total Viewing Universe tuned to a particular program
Program A
TV OFF
Program A
TV OFF
Program C
TV OFF
Program B
Program A B
TV OFF
Rating 20 10
Program C
TV OFF
20
134
> Media 101
Key Terms Ratings

A Few Top 20 Programs and Their Ratings
Everybody Loves Raymond
Super Millionaire
My Big Fat Obnoxious Fianc
11.5
8.7
12.2
Friends
14.6
CSI
17
*ratings based on Nielsen Top 20 135
> Media 101
Key Terms - Schedule Terms

Reach
- Number of different HHs/people exposed at least once to a commercial across a stated period of time Also called cume or unduplicated audience Maximum reach is 100% Individual program rating for a specific day also a measure of reach
Average Frequency - Average number of times a home/people are exposed to programs or commercials Reach & Average Frequency often detailed on both flowcharts and optimizer schedule runs Example: 100 GRP plan have a reach of 30 and average frequency of 3.3 Expressed as 30/3.3
136
> Media 101
Key Terms - Schedule Terms

Rating Point (GRP) Sum of the ratings delivered by a media vehicle, without regard to duplication, expressed as a % Reach x Frequency = GRPs GRPs indicate household ratings, TRPs refer to people rating points Terms are interchangeable as long as population base being used is cited Cost Per Point (CPP) Cost of purchasing one rating point Usually only used to measure costs for television and radio Calculated by taking cost per spot/rating point ($250 spot, 10 rating = $25cpp) Cost Per Thousand (CPM) Cost per one thousand individuals (or homes) delivered by a medium or media schedule Can be used for any medium (tv, radio, print, internet, etc.) To calculate CPM, divide the cost per media by the number of people delivered (expressed in thousands)
Gross
137
> Media 101
Key Terms - The Broadcast Calendar

Ad
Agencies & Advertisers use the Broadcast Calendar when referring to a media plan 4 Quarters in a calendar year Media purchased based on broadcast year (Q4 2003 Q3 2004) New contracts begin on networks with Advertisers
Weeks
are Monday Sunday Aligns with IRI weeks year 2004 begins with Monday, December 29th
Broadcast
138
> Media 101
How To Read a Flowchart
139
> Media 101
What is the Television Upfront?
Networks try to sell between 75%-80% of their commercial airtime ahead of the new fall TV season Takes place each year in May Networks announce new fall programming New TV season begins mid to late August Each network screens new programming for media buyers Negotiation period begins During negotiations buyers make a full-year schedule commitment for their clients advertising Clients that place advertising in the upfront have A guaranteed CPM (cost per thousand) The ability to cancel portion of the buys throughout the year Additional buys placed later in the year are called scatter buys Scatter usually more expensive Inventory availability depends on how well sales were during the upfront Some advertisers do not participate at all in upfront and only buy scatter and/or opportunistic network buys
140

Modeling Testing 101

Uploaded by

Copyright:

Available Formats

Modeling Testing 101

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Modeling Testing 101

Uploaded by

Copyright:

Available Formats

Modeling & Testing 101

Lead Identification & Qualification

Copyright 2005 Information Resources, Inc. Confidential and proprietary.

> Agenda: Modeling & Testing 101

Copyright 2005 Information Resources, Inc. Confidential and proprietary.

> Introduction: What Is Modeling & Testing?

What Are Modeling & Testing Analyses?

Copyright 2005 Information Resources, Inc. Confidential and proprietary.

> Introduction: What Is Modeling & Testing?

What Are Modeling & Testing Analyses?

Copyright 2005 Information Resources, Inc. Confidential and proprietary.

> Introduction: What Is Modeling & Testing?

Why Model Or Test?

Copyright 2005 Information Resources, Inc. Confidential and proprietary.

> Introduction: What Is Modeling & Testing?

Client Business Issues

Your client is under pressure to increase sales by 10% next year

Copyright 2005 Information Resources, Inc. Confidential and proprietary.

> Introduction: What Is Modeling & Testing?

Client Business Issues continued

Copyright 2005 Information Resources, Inc. Confidential and proprietary.

> Introduction: What Is Modeling & Testing?

Client Business Issues continued

the cost of being wrong high?

Would you use TESTING or MODELING to get to the answer?

Copyright 2005 Information Resources, Inc. Confidential and proprietary.

> Introduction: What Is Modeling & Testing?

When to Test vs. Model

Data Does Not Exist?

> Introduction: What Is Modeling & Testing?

When to Test vs. Model continued

Advanced Analytics Recommended?

> Introduction: What Is Modeling & Testing?

Store Data Regression Models

Store Group Analyses

Panel Data Models

Copyright 2005 Information Resources, Inc. Confidential and proprietary.

> Modeling: Drivers Suite of Products

Store Data Regression Models

Store Group Analyses

Panel Data Models

> Modeling: Drivers Suite of Products

1 Brand or 1 SKU Mix Planner, Mix Drivers CSI

Price Point / Gap Charts

Price Point / Gap Charts

Copyright 2005 Information Resources, Inc. Confidential and proprietary.

> Modeling: Drivers Suite of Products

Statistics Made Simple

> Modeling: Drivers Suite of Products

Statistics Made Simple continued

To what extent can I predict the change in sales as a function of change in

How well does the model predict the outcome? R2

> Modeling: Drivers Suite of Products

Store Data Regression Models

Store Group Analyses

Panel Data Models

> Modeling: Drivers Suite of Products

A multiple regression approach

Bayesian regression models

Incremental Impact of each marketing element