Experiments
Our new Experiments feature is here to help you get the most out of every email campaign you send! With Experiments, you can easily test different versions of your emails—whether it’s the subject line, the sender name, the send time, or even the content itself.
Think of it like this: instead of guessing what might work best, you’ll actually know what works, because your audience will show you with their opens and clicks.
Why Use Experiments?
Here are just a few reasons why using Experiments can seriously improve your email marketing results:
- Find what your audience loves: Understand what types of subject lines, content, or sender details drive better engagement.
- Avoid low-performance sends: Instead of sending one version to everyone, you test a few and only send the winning version to most of your audience.
- Make data-backed decisions: Stop guessing. Start sending smarter.
- Improve future campaigns: Use your learnings to keep growing and optimizing.
Setting up an Experiment
Setting up your first experiment is simple and intuitive. Just follow these steps:
Head over to Experiments and click Create your first experiment.
You’ll begin by naming your experiment—choose something descriptive so you can quickly identify what you’re testing (e.g. Subject Line Test – Summer Promo or Content Test – New Template).
You can test up to 8 combinations. You can mix them up if you see fit:
- Subject lines
- Sender names
- Send times
- Content variations
However, we strongly recommend testing only one variable at a time to clearly identify what’s making the impact—whether it’s opens or clicks.
Here is an example of testing subject lines. Choose the Subject Line option, then enter all the variations you want to test.
After that, select one sender name and email address that will stay consistent across all variations (since we’re only testing subject lines here). When you’re done, click Save and continue.
Next, set up the content for your campaign. Choose between:
- Drag & Drop Builder,
- Plain Text Email,
- Custom HTML.
Pick a template or start from scratch, then customize it as needed and save the changes.
In the next step, choose the groups and/or segments you want to include in your experiment. You can also exclude specific groups or segments to narrow down your audience, or simply select all active subscribers if you’d like to reach your full list.
This step ensures you’re targeting the right audience for meaningful results.
Once you select your recipients, scroll down to edit the experiment settings.
Test Group Size: Define what percentage of your total audience will receive the test. For example, if you’re sending to 1,000 subscribers, you might choose 20% (200 people) for testing.
Winning Group Size: The remaining percentage (e.g., 80%) will receive the winning variation once a winner is selected.
Winning Metric: Select what metric will determine the winning variation—opens or clicks. (See below for tips on which to use.)
Duration: Set how long you want the experiment to run. You can choose from hours, days, weeks, or even months—but we recommend keeping it under 24 hours for most experiments.
Once the test group receives all the selected combinations and the duration ends, Sender will automatically determine the winner based on your chosen metric and send it to the remaining (Winning) audience.
In the final step, you’ll see an overview of your experiment setup: all the subject lines, sender names, content variations, and testing settings.
If everything looks good, you can either send your experiment immediately or schedule it for a later time that best suits your audience.
Best Practices for Experiments
Which metric to choose?
Use Opens as your winning metric when testing:
Subject lines
Preview text
Sender names
Reply-to email addresses
Sending time
Use Clicks as your winning metric when testing:
Email content (e.g. layout, images, CTAs, offers)
Send time (if you want to see who actually engaged beyond just opening)
How long should your experiment run?
Duration depends on the size of your audience:
Up to 5,000 subscribers → Run for up to 24 hours
5,000–20,000 subscribers → Run for around 12 hours
20,000+ subscribers → Run for 4–8 hours (your list is big enough to get fast data)
IMPORTANT: Avoid low-activity periods, like weekends or late nights, when subscribers are less likely to engage. Testing during these times could give you misleading results.
How many variations should I test?
Up to 5,000 subs → Stick to 2 variations
5,000–20,000 subs → You can try 3 variations
20,000+ subs → You can go bigger and test more complex setups
The goal is to test efficiently—not overwhelm your audience or stretch the test too thin.
Start small, keep it simple, and use your learnings to send smarter campaigns every time.
That’s it!
If you got stuck on a specific task or can’t find a way to execute a particular job, contact our support team via LiveChat or [email protected] – we’re here to help 24/7.
