A/B and Multivariate Test Emails

About

Test emails allow users to identify the most effective ways to garner a Prospect’s interest with emails.

• A/B Test email functionality is available to Insightly users on all plans.
• Multivariate Test email functionality is available to Insightly users on Professional and Enterprise plans.

In this Article

About

How Does it Work?

Create a Test Email

How Does it Work? 

A/B and Multivariate Test emails allow users to send different versions of an email to different groups of Prospects with the goal of understanding what draws a Prospect's attention most effectively. 

• A/B Test Emails: An A/B Test Email allows users to test one variable with up to three combinations. Users can choose lists to send these emails to, but users cannot select which email each Prospect will receive. For example, users may want to test two subject lines to see which one Prospects are more likely to engage with.
• Multivariate Test Emails: Multivariate Test emails allow users to test multiple variables with up to eight combinations. Similarly to A/B Test Emails, users can choose lists to participate in a Multivariate Test but cannot choose which specific email combination each recipient will receive. For example, users may want to test two subject lines and two different sender names to see which one Prospects are more likely to engage with.

When utilizing either A/B Test Emails or Multivariate Test Emails, Insightly account users cannot be added to static or dynamic lists for these test emails. If an account wants to use their users for an A/B Test or a Multivariate test, Prospect records can be created for Insightly each account user and those Prospect records can then be added to either a static or dynamic list as applicable for the testing. ​​

When using test emails, the following variables can be tested by creating different variations of each variable and the email variation that is the most successful will be the “winner”:

• Subject Line: Experimenting with different wording, limited time offers, or emojis can be used to test what kind of subject line Prospects respond to best.
• From Name: Test emails can be used to decipher whether Prospects are more receptive to emails coming from a company or from the name of an employee. Different test emails can be created with various combinations of the sender name and the sender email address to test which is the most successful.
• Content: Different email templates or variations of the same email template can be tested to see which template garners more engagement.
• Send Time: Sending emails on different days and at different times can be done to test when Prospects are most likely to open an email. It should be noted that there is no “winner” if send time is selected as a tested variable.

When sending test emails, the email status can be reviewed to determine where in the process the test is. The status can be found within the test email record:

• Draft: If the test email hasn’t been sent or scheduled, the status will be Draft.
• Scheduled: If the test email is scheduled to start but no emails have been sent yet, the status will be Scheduled. 
• Sending: Once the first test email has been sent the status will be Sending. 
• Determining Winner: This status indicates that the last test email has been sent. The winning combination is chosen automatically based on the email open rate or the click rate of links inside the email (this is configured when a test email is created). 
  • When testing send times, all email recipients will be scheduled to a receive a test combination. 
  • If send times is not being tested, Insightly will allow users to send their test emails to a fraction of their Prospects and when a winner is determined, the winning combination will be sent to the remaining percentage of Prospects. 
• Sent: When all test emails have been sent (including the winning email), the status will be Sent. 

Create a Test Email

1. Navigate to Emails located via the objects on the left side of Insightly Marketing.
2. Click the drop-down arrow to next to the New Email button. Select either New A/B Test Email or New Multivariate Test Email.
   1. Enter the recipients information via the Recipients step and click Next when complete:

      Field	Input
      Email Name	Enter a unique name for the email.
      Folder	Assign the email to a folder. Efficient categorization of emails ensures that they are easy to locate.
      Recipient Lists	Search for and select the applicable recipient list(s).
      Suppression Lists	Search for and select the applicable suppression list(s). If a prospect is on both a recipient list and a suppression list, the suppression list takes precedence.

   2. Variables step, select which variables to test and click Next when completed. 
      1. Choosing any of the variables will display a summary of the test in the Email Test Summary section. The plus and minus buttons on the selected variable can be used to increase the number of email combinations.
         • A/B Test Emails: One variable with up to 3 combinations can be selected. 
         • Multivariate Test Emails: Multiple variables with up to 8 combinations can be selected. 
      2. Use the slider below the Email Test Summary section to determine the percentage of recipients that will receive the test combinations. 
      3. Determine how the winning email will be chosen – either by email open rate or email click rate. A period of either hours or days can be applied as the wait time until the winning email is determined. For example, if four hours is selected, Insightly will wait until 4 hours after the emails have been sent before deciding the winner. This wait time determines when the winning email combination is sent to the remainder of the list (i.e., if a whole day selected as the wait time, the remainder of the list will not be sent the email until the next day).  
   3. Apply the test email content via the Setup step. The options available in this section will display based on the variable(s) chosen to be tested via the Variable step. When the email content has been applied, click the Next button. 
   4. Choose a template to use for the test email via the Content step. Various prefabricated templates are available for use. Custom templates (created with the Designer Tool) can be found at the end of the prefabricated template list. 
      1. Once a template is chosen, it can be customized using the Designer Tool. When customization is complete, click Save Design and then click Next to continue to the next step in the creation process.
   5. The Summary step will display a brief summary of the test email. To send the test email, click Send. 
   6. The Finish step will display and Start Campaign can be clicked to complete the creation and sending of the test email. 
      1. If changes need to be made to a test email, the test email record can be opened where the Actions drop-down can be clicked and Edit This Email can be selected. Test emails cannot be edited once they have been sent.
         
   7. The winner of a test email will appear in the Details tab of the test email record. To review the results of the test email, utilize the Statistics tab of the test email record and use the Combination drop-down to see how each set of variations performed. 
      1. If no winning criteria is met within the test period (defined via the Variables step of the test email creation process), the very first combination will be declared as the winner and the winning segment will receive that email.