tag:blogger.com,1999:blog-8226170.post114299541060773989..comments2009-08-11T16:12:20.520-04:00Chris D'Alessandrohttp://www.blogger.com/profile/15362245792942554787noreply@blogger.comBlogger3125tag:blogger.com,1999:blog-8226170.post-1143719494281340172006-03-30T06:51:00.000-05:002006-03-30T06:51:00.000-05:00Hi Chris,<BR/><BR/>Great to hear more and more analysts are starting to use multi-variate tests. I agree with Dave: If you use specialized software you can make use of some great statistics that take care of the fact that you don't have to test all 75 iterations (or treatments) to get an answer on how well the different creatives (attributes) perform and what the interactions are.<BR/><BR/>There are several multi-variate test suppliers like Memetrics, SiteSpect, Optimost, Offermatica to name but a few.<BR/><BR/>Another comment I would like to make is that in my experience with multi-variate testing (about 2 years) is that you also need sound experimental design to get the data you want about a banner, page or AdWord. Because a lot of my clients come up with not a multi-variate test but a multiple A/B test in one experiment.Eelco van Kuikhttp://www.arroba.nlnoreply@blogger.comtag:blogger.com,1999:blog-8226170.post-1143719210954608512006-03-30T06:46:00.000-05:002006-03-30T06:46:00.000-05:00Hi Chris,<BR/><BR/>Great to hear more and more analysts are starting to use multi-variate tests. I agree with Dave: If you use specialized software you can make use of some great statistics that take care of the fact that you don't have to test all 75 iterations (or treatments) to get an answer on how well the different creatives (attributes) perform and what the interactions are.<BR/><BR/>There are several multi-variate test suppliers like Memetrics, SiteSpect, Optimost, Offermatica to name but a few.<BR/><BR/>My experience with multi-variate testing (about 2 years) is that you also need sound experimental design to get the data about how well a banner, landing page or AdWord will perform.Eelco van Kuikhttp://www.arroba.nlnoreply@blogger.comtag:blogger.com,1999:blog-8226170.post-1143003591819039032006-03-21T23:59:00.000-05:002006-03-21T23:59:00.000-05:00Hey Chris, a comment running 75 creatives, each 1.25% of the time, for a month to get actionable data...<BR/><BR/>Am I understanding you correctly - you're changing just the <I>call to action</I> text, and you have 75 different possible versions?<BR/><BR/>Assuming what you're really doing is changing the call to action text <I> plus</I> changing several other variables such as graphic image, font size, CTA placement within the banner, etc., then you might consider performing a regression on response rates across each of your ~75 combinations.<BR/><BR/>Regression (ANOVA) is part and parcel of multivariate testing (aka multivariable aka multi-variant) and tells you which of those overall variables (CTA vs. image, for example) was influencial on the response (banner CTR).<BR/><BR/>Even with smaller sample sizes you'll get the benefit of seeing which variables were the "levers" that influence response. And understanding these levers early on in the test (say 2 weeks into it) will enable you to pare down your list of ~75 and concentrate testing efforts on fewer, more meaningful combinations. The more iterative pruning you can do, the faster you will converge on the optimal creative combination.Dave from SiteSpect Multivariate Testinghttp://www.sitespect.comnoreply@blogger.com