How to Run A/B Tests on Emails
A/B testing helps you find what resonates with your audience. OutSends lets you test subject lines, email content, send times, and more — and automatically sends the winning variant to the rest of your list.
What you'll need
Before you start, make sure you have the following ready.
- An OutSends account
- A campaign with at least 500 contacts
- Two or more subject lines or email variants
Step-by-step guide
Follow these steps to complete the setup.
Enable A/B test mode
In campaign creation, toggle "A/B Test". Choose what you want to test (subject line, content, from name, or send time).
Create your variants
Write 2-5 variants. For subject lines, make them distinctly different — don't just change one word.
Set test size and duration
Choose what percentage of your list gets the test (20% is standard). Set the test duration (2-6 hours recommended).
Monitor test results
OutSends tracks open and click rates per variant in real time. Once a winner is statistically significant, it's declared.
Winner sends automatically
The winning variant automatically sends to the remaining 80% of your list. No manual intervention needed.
Tips for best results
Pro tips to get the most out of this setup.
- Test one variable at a time — testing subject AND content together won't tell you what worked
- Use a 90% statistical significance threshold before declaring a winner
- Don't end tests too early — 2 hours minimum for meaningful results
Frequently asked questions
What sample size do I need for A/B testing?
At least 100 contacts per variant for statistically significant results. 500+ is better.
Can I A/B test more than one variable?
You can, but it requires multivariate testing, which needs a much larger sample size. Stick to one variable per test.
What happens if there's no clear winner?
OutSends picks the variant with the highest absolute performance and sends it to the rest of your list.
Related guides
Explore more how-to guides for OutSends.
Ready to get started?
Create a workspace, connect a sender, and launch your first campaign — free, no credit card.