A/B testing, split testing or bucket testing is a method of marketing testing by which a baseline control sample is compared to a variety of single-variable test samples in order to improve response rates. A classic direct mail tactic, this method has been recently adopted within the interactive space to test tactics such as landing pages, banner ads and emails. The questions of this can happen in WordPress is asked often and rarely answered. The SES split testing plugin for WordPress allows you to effectively conduct A/B test on your WordPress sites. Assuming Google Analytics code is already integrated in the pages, the following four simple steps will help you get rolling with the A/B (/C etc) tests:
More often than not, you want to test two versions of one theme with some small changes. Assuming your theme is in the folder wp-content/themes/mytheme, take a copy of your theme and put it in fresh folder, e.g. wp-content/themes/mytheme2. Open up the style.css file in the second folder, and change the Theme Name, e.g.
/* Theme Name: Theme */
/* Theme Name: Theme (v2) */
Now install and activated the plugin, and then go to the settings page (Settings->SES Theme Split Test). The plugin will list all of your available themes, and you can choose which ones you want to be part of the test. Once you have saved the settings, the split testing will kick in. Visitors will be randomly assigned one of the chosen themes, and they’ll receive that theme any time they visit in the next 30 days. Not only that, but their visit will be tagged with a special variable that you can pick up on in Google Analytics.
Google Analytics lets you define custom segments (Ways of grouping customers). In this example we want to group customers according to which theme they were shown and then we can measure our chosen metric (Purchase / sign-up / page views – whatever) for the two groups and compare them. To get started, you probably want to wait 24 hours so that Google starts to receive the tagging data.
Log into your analytics account, and click on the “All Visits” dropdown (Top-right, just above the date selector), and then choose “Create a new advanced segment”. Then, you want to expand the “Visitors” dropdown under “Dimensions”
Scroll down until you find “User Defined Value” and drag it onto the right hand box that says “dimension or metric”. If Analytics has got your tagging data (See – I told you to leave it 24 hours!) then you’ll get to choose from the values being passed, e.g.
Create a segment for each of your themes, and give them a meaningful name, then you’ll be able to select them as custom segments in your reports. Now it’s just time to wait, see what happens, then decide. Once you’ve got a winner then just set that as your blog’s standard theme in the normal way, and either deactivate the plugin, or unselect the themes in the settings and users will see the blog default.
(excerpts of this article have been taken from leewillis’s blog)