Creating and using A/B tests in your beehiiv posts

With beehiiv paid plans, you can test different subject lines with a small portion of your audience before sending the final email to all subscribers. This helps you choose the most effective subject line for your audience, increasing open rates and providing insights into your audience’s preferences. 

At the end of the A/B test, the subject line that performs best will be used for the rest of your send, maximizing the reach of your post based on what resonates most with your audience.

Quick tips to get you started

• A/B testing vs. test emails: A/B testing is different from sending a test email, which lets you review how the entire post appears by sending it to yourself or your team. A/B testing, on the other hand, focuses on testing different subject lines with a portion of your audience to see which performs better.
  
  
• Where to find A/B testing options: You can set up A/B testing on the Email page as you move through the post creation process, as outlined below.
  
  
• Customizations for A/B tests: You can test up to 4 subject lines over a designated time period. Additionally, you can control the percentage of your audience that will receive the A/B test. During the test period, changes cannot be made.
  
  
• Determining the winning subject line: Once the test is complete, beehiiv will assess if there is a statistically significant winner. If no clear winner emerges, the subject line with the most opens will be chosen. This winning subject line will then be sent to the rest of your designated audience. In case of a tie, the original subject line will be sent to all subscribers (as opposed to the variants that were tested against it).
  
  
• Web post versions: If your post is being published to both web and email, the web version will be available at the start of the A/B test period (at the time of original publication). 

---

How to create an A/B test 

1. From the left panel Dashboard, click Start Writing, or open a post that you would like to edit.
   
   
   
   
   
   
2. The Post Builder will open on the Compose page. Draft your newsletter content as desired.
   
   Once you're done, click Next to proceed to the Audience page, where you’ll select the audience you want to send your post to. After choosing your audience, click Next again to continue.
   
   
   
   
   
   
3. Or, if your post is already drafted and the audience has been selected, you can go directly to the Email page using the top menu options.
   
   Under Sending details, you'll see your subject line and preview text, which default to your post title and subtitle unless you choose to modify them. To set up an A/B test, simply toggle on the A/B test option.
   
   
   
   
   
   
4. The first subject line, or Subject line A, will default to your post title (if one was added). Otherwise, write your first subject line option into field A. You can then add up to three additional subject line variants.
   
   To do this, click Add new subject line and type each subject line into the input box, then click outside the box to add the variant.
   
   
   
   
   
   
5. If you want to delete any subject lines you've added, simply click the trashcan icon next to the specific line. Keep in mind, you must have at least one subject line for your post and at least one variant for the A/B test to proceed.
   
   
   
   
   
   
6. Set the duration for your A/B test. By default, the test will last 60 minutes, but you can choose any time frame between 5 and 240 minutes. During this period, we'll track performance to identify the best-performing subject line. After the test ends, the winning subject line will automatically be sent to the rest of your designated audience.
   
   
   
   
   
   
   Tech Note: Once you’ve set the duration and begun the A/B test, it cannot be ended early.

    

7. Choose the percentage of subscribers to include in your A/B test. By default, the test group is set to 10% of your audience, but you can adjust this. Based on the percentage you select, we’ll show you the exact number of subscribers in that group. This test group will then be equally divided, with each smaller group receiving one of your subject line variations.
   
   
   
   
   
   
8. Below the A/B test options, you’ll find your preview text. By default, this text matches your post subtitle unless you choose to modify it.
   
   Also keep in mind that the preview text remains consistent across all subject line variations during the A/B test. Changes are synced automatically.
   
   
   
   
   
   
9. Click Next to proceed to the Web page, where you can configure SEO settings and web details for your post. 
   
   
   
   
   
   
10. After adding the Web details for your post, click Review to move onto the final step.
    
    
    
    
    
    
11. On the Review page, you'll find three key sections under Review and Publish. Open the A/B test section to double-check that everything is correct. When ready, click Schedule to finalize and send.
    
    
    

---

How to see your A/B test results

1. From the left panel Dashboard, click on Posts and open the post for which you want to see the results.
   
   
   
   
   
   
2. You’ll be presented with multiple tabs that provide different analytics for your post. Click on the A/B Test tab.
   
   
   
   
   
   
3. Here, you can view the performance of each subject line variant and see which one emerged as the winner.