A/B Testing Server Side using Shopify and Google Optimize

A tutorial to set up server-side google optimize A/B tests in Shopify.

Overview:

We want to manually put people in the experiment and serve two different experiences for a page, but still use google optimize to track the results.

Server-side A/B tests in google optimize are possible. This guide assumes you're using the automatic implementation of Google Analytics, and adding the google optimize module through the additional javascript section in Online Store > Preferences.

The only challenges here are we need to determine whether a visitor is in the control group or the variant group prior to the google analytics module loading in, and "remember" which group a user is in so that they have the same experience if they return later. We'll use client-side javascript for the former, and cookies for the latter.

In this example, we'll be setting up an A/B test to see whether or not offering free gifts based on the cart value increase the likelihood that users complete the checkout process.

First, we'll need to set up the experiment in optimize. The important bit here is that we need to set the targeting rules such that it will never fire. The reason for this is because we'll be adding people manually to the experiment, so we never want the optimize module itself to add anyone to it too. Do this by setting the targeting rule to a non-existent URL, like 'SERVER_SIDE'. Then, start the experiment, and you'll see the experiment ID appear in the sidebar.

Next, we'll need to add a snippet before the {{content_for_header}} liquid tag in the head tag. This snippet is responsible for deciding whether a given user is in the control group or the variant group, and setting a cookie accordingly.

Create a snippet with the following code (we named it 'cro_vars'):

<script>
//semi-colons, commas, equals signs, and white spaces are not allowed characters.
//get set delte from https://plainjs.com/javascript/utilities/set-cookie-get-cookie-and-delete-cookie-5/
function getCookie(name) {
var v = document.cookie.match('(^|;) ?' + name + '=([^;]*)(;|$)');
return v ? v[2] : null;
}
function setCookie(name, value, days) {
var d = new Date;
d.setTime(d.getTime() + 24*60*60*1000*days);
document.cookie = name + "=" + value + ";path=/;expires=" + d.toGMTString();
}
function deleteCookie(name) { setCookie(name, '', -1); }


if(getCookie('cro_test') == null){
//FIRST SESSION - CHOOSE & SET COOKIE
console.log("first timer! lets set a cookie.");

if (Math.random() >= 0.5) {
//variant
console.log('welcome to the experiment, muahaha.');
setCookie('cro_test', 'popup', 100);
} else {
//control
console.log('sorry you will be the control');
setCookie('cro_test', 'nopopup', 100);
}

}

if(getCookie('cro_test')){
//RETURNING SESSION - LOOK UP GROUP
console.log("welcome back, lets see if you're in an experiment");

if(getCookie('cro_test') == 'popup'){
console.log('you get a free gift!');
atlas = 'popup';

} else {
console.log('shhh. behave as normal');
atlas = 'nopopup';
}
}
</script>

and add this snippet to theme.liquid before {{content_for_header}}, (in our case: {% include 'cro_vars' %}). This script contains function definitions to set, read, and delete cookies (in "vanilla" or plain javascript - so jQuery is not needed prior to code execution).  First, it checks for the existence of a cookie, and if a cookie is not detected, then it flips a coin (Math.random), and then sets a cookie depending on whether they're in the control group or the test group. This one is set for a 50/50 split, but it could be changed to any other percentage by changing the decimal in the first if/else block. Next, it checks for the existence of a cookie, and sets a global variable according to the result. The reason for the separation is that we want it to always check for the cookie to decide how to act, regardless if the cookie was set during that session or a previous one. 

To recap, up to this point we have a cookie being created or read, and a corresponding value stored in a global variable - now we just need to check for that variable, and then link the session to the corresponding segment in Google Optimize so that we can measure the results of the experiment both in Optimize, and in Google Analytics. To accomplish this, we'll leverage the additional javascript field in Online Store > Preferences to add some javascript which will be injected in the {{ content_for_header }} tag. This javascript checks the value of the global variable initialized by the snippet, and fires a google property accordingly:

if(atlas == 'popup'){
ga('set', 'exp', 'B2F6VqZsQGbrDy_5F8l6Zg.1');
console.log('set exp to variant');
}
if(atlas == 'nopopup') {
ga('set', 'exp', 'B2F6VqZsQxYrDy_5F8l6Zg.0');
console.log('set exp to control');
}

The long number which is the last argument to the ga function is the experiment ID, and the integer after the period corresponds to the control group or experiment group (for A/B/n tests there would be more). When this fires, it will trigger the optimize module, which will run any scripts or apply any style changes associated with the experiment. One thing to note: the experiment ID isn't generated until the experiment is started within Google Optimize.

And basically that's all you need! The term "server-side" is used rather loosely here, as it's just the configuration of the experiment being handled serverside (client-side javascript determines which variant of the experiment a user should be in, but server-side in the sense that the needed javascript is outputted using liquid code and logic, and the functions to update the GA module are automatically injected by the scripts aggregator). A full server-side implementation would roll the dice server side, and allow different templates, sections, or snippets to be served accordingly. The chief obstacle is a lack of a random number generator in the Liquid templating engine, though I do think a suitably-random number generator could be created leveraging the server's timestamp, the liquid modulo functions, and perhaps a language .json file to use as a hash or lookup table. 

If anyone has experience generating random numbers server-side in Liquid, I'd love to hear about your approach and results!