A/B testing is a simple yet reliable method to verify whether your online store's or website's overall efficiency will increase if you change something in their design or content. A/B testing means to alternately display one (А) or other (B) version of some web page to website visitors and the subsequent analysis of each version's qualitative and quantitative parameters such as conversions , subscriptions, orders, visit duration, etc. For example, you may use A/B-testing to compare the conversion rate and the sales volume dependency on specific products displayed in the homepage. Read more about A/B testing on Wikipedia: http://en.wikipedia.org/wiki/A/B_testing.
There are various ways you can conduct A/B testing using Google Analytics: by means of their Experiments feature or using custom variables. This article describes an example of organizing A/B testing in an online store powered by Shop-Script 5 by means of custom variables in Google Analytics.
As an example, let us compare sales volumes in an online store with different product lists displayed in the homepage (“special offers”). To do so, we shall create an additional product list to compare it with the one already available in the store. To obtain more precise results, it is better to use static lists whose contents will not change during the experiment.
1. Add Google Analytics tracking code
In order to analyze test results, enable integration with Google Analytics for your website.
2. Start the test
Enable random number generator
To begin A/B testing, add the code snippet provided below to the very beginning of the desired design template forming the page you need to test. If you are testing a simple info page, add this snippet to its source code. In our example we shall add it to template file home.html:
{$AB_version = $wa->storage('AB_version')}
{if empty($AB_version)}
{$AB_version = mt_rand(1, 2)}
{$wa->storage('AB_version', $AB_version)}
{/if}This code reads the value of field AB_version in the visitor's PHP session. If there is no such field in the session (i.e. no page version has been yet defined for the current visitor), then a random value is calculated for him, either 1 or 2, and written to the session. Now we know which page version will be shown to this specific visitor (to be more exact, the value of the $AB_version variable, which is equal to 1 or 2).
Important note: Add the same code to the source of template file index.html. This is necessary for the $AB_version variable to be also defined within the layout's common template in addition to the specific template you are testing.
Add condition for displaying different page versions
Suppose you have some product list, with “promo” id, displayed in the home page, and you have decided to check whether you can increase your sales by replacing it with a different product list, with “promo_test” id.
To do so, use the value of the $AB_version variable containing a randomly generated number to create a condition displaying different HTML content (i.e., product lists) to different visitors:
{if $AB_version == 1}
{include file="list-thumbs.html" products=$wa->shop->productSet('promo')}
{else}
{include file="list-thumbs.html" products=$wa->shop->productSet('promo_test')}
{/if}Then use a similar condition in file index.html to add calls of JavaScript method _setCustomVar:
{if $AB_version == 1}
<script type='text/javascript'>
var _gaq = _gaq || [];
_gaq.push(['_setCustomVar', 1, 'HomepageProductList', '<strong>Promo product list (old)</strong>', 2]);
</script>
{else}
<script type='text/javascript'>
var _gaq = _gaq || [];
_gaq.push(['_setCustomVar', 1, 'HomepageProductList', '<strong>Promo product list (new)</strong>', 2]);
</script>
{/if}Adding such snippets of JavaScript code assigning different values to variable HomepageProductList (“Promo product list (old)” and “Promo product list (new)”) is necessary to collect statistics for your Google Analytics account (ensure that integration with this service is enabled in the Site app as described above).
If, in parallel with Google Analytics powered A/B testing, you also carry out similar tests using the built-in Shop-Script testing tool, then instead of the random number generator code condition described above you may use the A/B test condition code which is automatically offered when you use the built-in tool in Shop-Script admin backend.
The latter condition containing JavaScript code must be added before the following line:
{$wa->headJs()}This line includes Google Analytics code in the storefront layout, which also contains calls of _trackPageview method, and it must be called after method <code>_setCustomVar for statistical data to be registered correctly.What means each of the parameters in those JavaScript code lines:
_gaq.push(['<strong>_setCustomVar</strong>', <strong>1</strong>, '<strong>HomepageProductList</strong>', '<strong>Promo product list (new)</strong>', <strong>2</strong>]);
_setCustomVar: This is a system parameter, write it exactly as shown here.
1: Slot number. Custom variables can be distributed among several slots in Google Analytics; for the sake of example we are using slot 1.
HomepageProductList: Variable name. It must be the same for each of the versions being tested; you may create the name for a variable at your discretion.
Promo product list (old) and Promo product list (new): Variable name. It must be different for each version being tested to enable you to view separate statistical information in Google Analytics.
2: Variable scope. In our example we are using scope 2, i.e. the variable is valid within the PHP session lifetime.
Once this has been done, different versions of your store's homepage will be displayed to different visitors:
3. Track statistics in Google Analytics
in order to compare the efficiency of each page version participating in the test, set up segments in your Google Analytics account for each value of your custom variable, as described below.
Segment is a portion of statistical data filtered by various criteria; in our example segments will contain data filtered by the values of the custom variable HomepageProductList defined in online store's template: “Promo product list (old)” and “Promo product list (new)”).
- Choose the report within which you need to view A/B test related data; e.g., “Behavior → Overview”.
- Open the segment-setup section (using a button at the top of the page) and add a new segment.
- Enter the segment name so that clearly describes the corresponding page version; e.g., you may simply copy the variable value here, i.e. “Promo product list (old)” or “Promo product list (new)”.
- As the segment-forming criterion choose option “Conditions”, parameter “Custom variable (value 01) → contains”, and paste one of your custom variable values in the provided text field.
Repeat these steps to add another segment—for the second variable value.
Your custom segments based on the specified variable values are now available in section “Standard Reports → Audience → Custom → Custom Variables”.
4. Apply testing results
Compare the data displayed for each variable value, i.e. each of the tested page versions. If some of the versions has significantly outperformed the other and you decide to keep it as the main version, then remove all code related to A/B testing from the storefront template by leaving only the winner's source code, as described below:
- remove the random number generator
- replace the entire A/B testing condition with the winning version's source code in the template; in our example, remove the following section:
{if $AB_version == 1} {include file="list-thumbs.html" products=$wa->shop->productSet('promo')} {* inefficient version *} <script type='text/javascript'> _gaq.push(['_setCustomVar', 1, 'HomepageProductList', 'Promo product list (old)', 2]); </script> {else} <strong>{include file="list-thumbs.html" products=$wa->shop->productSet('promo_test')}</strong> {* winning version *} <script type='text/javascript'> _gaq.push(['_setCustomVar', 1, 'HomepageProductList', 'Promo product list (new)', 2]); </script> {/if}Instead of the removed code, add the source code of the more efficient page version; e.g.:{include file="list-thumbs.html" products=$wa->shop->productSet('promo_test')}
Tips
- Correct A/B testing should be run by alternately displaying various page versions during one common period of time. Incorrect A/B testing might include displaying of some page version for some time and measuring conversions, and then switching to another page version and again measuring conversions during the same period of time. During different periods of testing individual page versions, very different visitors (e.g., by source and quantity) may come to the website, and it is, therefore, incorrect to compare them in such a way.
- You may compare more than only 2 versions of a page, this is called A/B/N testing. To do so,change the number of page versions being tested in the random number generator code:
{$AB_version = $wa->storage('AB_version')} {if empty($AB_version)} {$AB_version = mt_rand(1, 3)} {* specify the desired number of page version; e.g., 3 *} {$wa->storage('AB_version', $AB_version)} {/if}Accordingly, the condition in your HTML template will slightly change, too:{if $AB_version == 1} ...paste current HTML code here... <script type='text/javascript'> _gaq.push(['_setCustomVar', 1, 'HomepageProductList', '<strong>Version 1</strong>', 2]); </script> {elseif $AB_version == 2} ...add first experimental page version here... <script type='text/javascript'> _gaq.push(['_setCustomVar', 1, 'HomepageProductList', '<strong>Version 2</strong>', 2]); </script> {else} ...add second experimental page version here... <script type='text/javascript'> _gaq.push(['_setCustomVar', 1, 'HomepageProductList', '<strong>Version 3</strong>', 2]); </script> {/if} - Avoid making changes to your marketing campaigns or website structure during an A/B test as this may distort the data you will obtain and prevent you from getting a clear picture of your website's efficiency.
- Wait until you have collected sufficient amount of statistical information from your tests; avoid making decisions based on data obtained during a short period of time, which will be incorrectly affected by the random number generator.
- Make sure that you test only one website element at a time. When you test multiple elements at the same time, each of them may have unobvious effect on the final results in combination with others (e.g., if you try to test different designs of the “Add to cart” button and the category tree). This way, you cannot obtain accurate testing results for each such element.
- If you feel that you need some confirmation that an A/B test, once completed, was indeed correct, you may simply repeat it at a later time. Should you obtain the same results again, you may be sure that the winning page version indeed has the right to remain on your website.
1 comment