Observation and Design

Richard Stowey's localised view of the world.

Flickr LinkedIn Twitter RSS Feed

The Importance of MVT (Multivariate testing) in Marketing Campaigns

October 24th, 2010 @ 10:44 GMT by Richard Stowey


The quantitative results of multivariate testing are extremely useful to improve the performance of a marketing campaign. It helps turn a great design into a well performing campaign which delivers important and valuable results.

Multivariate or A-B testing is generally used as a method of comparing two or more different versions of a web page on a live web site. This technique can be used for many different comparisons:

  • Completely different website designs
  • Slight differences in layout or page design
  • Colour variations
  • Iterative link enhancement
  • Improve performance of interactive elements
  • Imrove the buying process or more complicated set of pages.
  • Guage performance of different images
  • Headline and body copy

Google is well known for testing 41 shades of blue against each other to acheive the optimum performing page. The BBC uses multivariate testing to increase the performance of links. The quantitative results are hard to argue when the testing is done with live users, on a live website and within the actual space it’s going to be added.

But, what else can we use it for?

The process which leads people into a website, the advertising and marketing campaigns, often features some very excellent and creative work to lure people into a website or landing page. These campaigns often contain a lot of creative input and value, but within the short space of time they run, MVT is often overlooked. The use of multivariate testing could improve the performance of marketing campaigns by using multiple versions all the way along the campaign route:

  • Online Ads
    • Different calls to action (CTA) (copy and prominance)
    • CTA’s in different position (per ad format)
    • CTA’s in different position in timeline?
  • HTML Email
    • Different CTA’s
    • Layout
    • Use of images
    • Use of copy
    • Type of calls to action
  • Landing Page
    • Different CTA’s
    • Differing number of form fields
  • Final Destination
    • Decrease drop-off rate after download / sign up / purchase
    • Is the CTA thank you page the home page?
    • Where do we ultimately want people to go?
    • Can the thank you page performance be improved?

This is mainly the digital process, but the techniques could also be used with press advertising. The call to action on an offline mailing could be split across two or more user groups in order to measure which one is more successful. Driving people to two entirely different pages, instead of driving people to one page with two versions.

Quantitative analysis of website desgn, development and the surrounding methods of acheiving a higher interest rate in the assets and collateral you are offering, is an extremely valuable and completely underrated method of acheiving a better success rate with a campaign.

MVT could even go as far as to measure the performance of Twitter posts (or other social media for that matter), and the content of those posts in their performance to deliver the right message, at the right time, to the right people, and engage people’s interest.

So, in summary…

Multivariate testing isn’t the most important aspect of campaign development, but it is the most underused and possibly undervalued. It’s often the part of the process which is handled by an internal team, for example at the BBC, Google or Moo.com where they have an ongoing concern for their continuing performance.

It could be used more widely in marketing campaigns, allowing collateral to start with multiple versions of ads and landing pages, and then remove the underperforming version once enough time has passed to realise their performance.

MVT is important within website design and development and has great potential to turn around underperforming elements, assets, pages, ads and campaigns.