Thursday, March 6, 2008

TA Tone Omniture Summit 2008 Wendsday 3:30-4:45 BreakOut | Accelerate Your Path to Success with Multivariate and A/B Testing

The second Break out session
Josh Friedman DELL Case study
Lily Chiu Omniture Consultant

Today’s Online market -> paid search, email, display and costs are increasing. All traffic hits your site and all of your traffic gets the same experience even though they are acquired from many different sources.

The Test and Target funnel brings traffic and places them in relevant places on your site. We know a lot about visitors just because of the time location and geography of the visitor. Not to mention their behavior of your site.
Customizing landing pages is just the tip of the iceberg you can optimize your whole site.

“I have data now what?” You must be able to touch the site. In order to test and experiment with the results. If IT has you locked down you need to change it.
Segmentation is huge. It is like reaching up through the bottom of the funnel and pulling your customers through the funnel.

High frequency customers should have best conversion so don’t keep showing what they have already seen. Mix it up.
Tips
Start with something that simple both technically and politically. The hero banner is often a hot potato.
Know your goal and define the metrics and KPI’s
Define your question in mind (now I know that it is more effective to promote recreation than keynote speakers)
Test brand content vs hero image.
Test removing extra copy and navigation from the page
If you are paying for those trust symbols then move them above the fold and plaing them close to the call to action

Multivariate testing: Test Multiple elements simultaneously
Measure the influence and lift associated with each elements
From a/b to multivariate tests 2 elements 2 alternatives = 4
10 elements x 2 alternatives = 1024 permutations to test! Depending on
Smallest fractional factorials. Get speed and control change content with out IT
Think different when designing alternatives, What Am I trying to optimize. Define how each test will be defined as a winner.
Do not design sub optimal alternatives (to much alike)
You can learn from tests that fail.
Try removing Left navigation
Static vs dynamic content
Present vs non present
Standard vs conceptual merchandising

Filter test results by important segments and see if they differ one from another
Test important elements around key form
Test call to action in various forms fonts, colors and types of call s to action

Dell pilot summary
5 MVT over two weeks
Test Goals: Increase Revenue per Visitor (RPV) for visitors coming from high impact placement flash add on MSN homepage 81 combination and 9 recipies(through tuguchi) test ran on a single day.
Received a +28% Revenue per visit best recipy was most effective in the afternoon house: TRAFFIC SOURCES AFFECT BEHAVIOR
Test Objectives Increase RPV
Test Elements:
1. Banner Locaton, 2. Banner messaging, 3 call to action location,4. call to action nomenclature

What we have learned:
1. Stop Guessing
2. Relevance is key ( what page have they just seen, where are you going what do you know about
3. Frequency matters. You get more than just lift you get learning about your customers as well
4. Campaign optimization rules
5. Crawl, Walk, Run. Start an iterative process.


** You can have two id’s one for the visitor and 1 for the visit so that any visitor gets a consistent experience

No comments: