# Understanding your A/B test results

### What does ‘Chance to Beat Original’ mean?

Chance to beat original (CTBO) is the likelihood that the variation will outperform the original.

It determines statistical significance by answering the question:

*Is the difference observed between the original and variation greater than a difference due to random chance?*

### What does a CTBO of 50% mean?

A CTBO of 50% means that the variation will outperform the original 50% of the time, which is the same as random chance, therefore, there is **no difference** between your original and variation.

The closer your CTBO is to 50% (e.g., 40% or 60%), the less significant the difference between your original and variation.

The further your CTBO is from 50% (e.g., 5% and 90%), the more likely it is that there is a true difference between your original and variation.

### What does “Original is beating Variation” or vise versa mean?

Customer.io uses a significance level of 95%, meaning that CTBO has to be less than 5% or greater than 95% in order for us to report that one version is outperforming another.

If CTBO is > 95%, you can infer that your variation is outperforming your original.

If CTBO is < 5%, you can infer that your original is outperforming your variation.

What does “Not significant, need more data” mean? If your CTBO is between 5% and 95%, it does not meet our threshold for statistical significance. If your sent volume is low, gathering more data may prove that a significant difference exists.

### How is CTBO calculated?

There are 3 steps to calculating CTBO:

**1. Calculate the standard error for the original and variation.**

**2. Calculate the Z-score for the original and variation.**

**3. Based on the Z-score use a statistics table to determine the p-value and CTBO.**