How to split the traffic in an A/B Test
Choosing whether to split the test when a user lands on your website vs. when they get exposed to your test can significantly impact how long your test takes to complete.
When I started my A/B testing, there were several things that I understood at a superficial level (stuff I had read in books or online videos). However, I was fortunate to have colleagues that helped me understand these ideas intuitively, including basic concepts behind test design. I wrote about this in another article titled ‘The intuition behind A/B Testing — A Primer for New Product Managers.’
Once I got past the basics, other techniques helped me optimize the design of my tests. In this article, I will talk about one of these techniques — Test Exposure.
Understanding “Test Exposure” is essential because it helps design the test in a manner that can help you get to results faster.
Let’s go shopping!
We’ll start with a simple example. Imagine your product is a website www.merchant.com that lets users buy furniture. The following diagram explains how the user may navigate through the various sections of the website.