Understanding Significance in Your Testing

After you made a chance to a certain ad and then conducted an A/B test. You might ask, “What’s next?” Well, you will need to determine if the change that was made provided a significant difference or not. That is when the p-value gives a ton of useful information to the researcher.

Source: Medium

What Is the P-Value?

The p-value is how researchers, companies, marketing analysts determine if the results are statistically significant. According to Marketing Mojo (Links to an external site.), “Statically significant means the relationship in the results did not occur by random chance.” If the p-value is less than 0.05 then we can determine that the results are statically significant. If the results lie greater than 0.05 then we must conclude that the results are not statically significant.


In conclusion, the p-value isn’t the ultimate end factor for your marketing experimentation. It is however a great indicator if the change in your ad provided significant results or it was merely luck. Other pieces such as open rates, views, click rates are all important pieces of information needed in order to make effective business decisions. Click here (Links to an external site.) to read my last week’s post on another type of experimentation, copy testing!

Leave a Comment

Your email address will not be published. Required fields are marked *