Menu
 

Newsletter Split Testing

Tune your newsletter for maximum impact with our split testing feature. You can build multiple email variations and compare the results to learn what works best for you. You can also send to a random sample of your audience first and use the results to make sure the highest-performing version goes to everyone else.

Split Testing

To create a split test, use the Add Variation button in your newsletter creation workflow:

Create Split Test

This creates a new email variation to customize. You can change just about everything: subject, layout, from address, or the main content. Give it a name so you can recognize it later.

Create Split Test

Sending Behavior

Newsletters with multiple email variations will be split evenly across all of them. For example, if you have two templates, each will go to a random 50% of your recipients.

We allow a multi-way split, not just A/B. You can create up to 8 variations, split evenly like the below screenshot (though you’ll perhaps want to name yours a little more creatively than we have here):

Newsletter split content

Metrics

Once your newsletter is sent, we’ll track all the metrics individually: opens, bounces, clicks, and conversions. You can view a summary for the entire newsletter or drill down into each variation.

Newsletter split metrics

Use these reports to see which variation performed better.

Send to Sample Percentage

One common workflow is to send a split test to a sample percentage of your recipients, decide which variation performs best, and then send the best-performing content to everyone else.

From the recipients page, click on “Send to sample percentage”, choose how many people to send to, and continue as normal.

Newsletter sample percentage

Note that this feature is only enabled for newsletters with at least 500 recipients. For fewer recipients, samples are not statistically useful.

Once you’ve sent to a sample, you may want to send another newsletter to the rest of your recipients. To do this, there are two options:

Option 1 (faster if you have a large number of people in your account):

From the Recipients page of your split test newsletter, export the list of people who didn’t receive your sample:

Then select the exported CSV for the second newsletter:

newsletter upload CSV option

Option 2:

Create a segment for people who have not been sent the original:

Newsletter not sent segment

Then add this segment to your original recipient settings for the follow-up newsletter:

Newsletter send to rest of recipients