Newsletter Split Testing
Tune your newsletter for maximum impact with our split testing feature. You can build multiple email variations and compare the results to learn what works best for you. You can also send to a random sample of your audience first and use the results to make sure the highest-performing version goes to everyone else.
To create a split test, use the Add Variation button in your newsletter creation workflow:
This creates a new email variation to customize. You can change just about everything: subject, layout, from address, or the main content. Give it a name so you can recognize it later.
Newsletters with multiple email variations will be split evenly across all of them. For example, if you have two templates, each will go to a random 50% of your recipients.
We allow a multi-way split, not just A/B. You can create up to 8 variations, split evenly like the below screenshot (though you’ll perhaps want to name yours a little more creatively than we have here):
Once your newsletter is sent, we’ll track all the metrics individually: opens, bounces, clicks, and conversions. You can view a summary for the entire newsletter or drill down into each variation.
Use these reports to see which variation performed better.
Send to Sample Percentage
One common workflow is to send a split test to a sample percentage of your recipients, decide which variation performs best, and then send the best-performing content to everyone else.
From the recipients page, click on “Send to sample percentage”, choose how many people to send to, and continue as normal.
Note that this feature is only enabled for newsletters with at least 500 recipients. For fewer recipients, samples are not statistically useful.
Once you’ve sent to a sample, you may want to send another newsletter to the rest of your recipients. To do this, there are two options:
Option 1 (faster if you have a large number of people in your account):
From the Recipients page of your split test newsletter, export the list of people who didn’t receive your sample:
Then select the exported CSV for the second newsletter:
Create a segment for people who have not been sent the original:
Then add this segment to your original recipient settings for the follow-up newsletter: