Testing your messages against control groups shows you which messages are driving real action. We’ll send your message to half of your intended audience, and hold back the message for the other half.
Then, you can easily see which group performed better. This will help you make informed decisions about which messages to keep, and which to refine.
Note: Control groups are available for ongoing messages. They’re not available for one-off messages. See our plans and pricing here for more details.
Create a control group
Once you’re finished writing your message, click Run a new test. Then, select Control group:
Note:
You can only set a control group when you’re drafting a message. Once a message is live, you can’t set a control group for it.
Control groups are available for all message channels (chat, post, email and push).
Now, once you’re happy with your message, set it live.
How control groups work
Next, we’ll split the audience of the message into two groups. Then, we’ll automatically assign 50% of your audience to the message and 50% to the control group. That means 50% of your audience won’t receive this message.
See the results of your control group experiment
Once your message is live, it’s easy to see the results of your control group experiment.
First, click on the Overview tab - here you’ll see the number of customers who received the message, and the percentage of those customers who completed your goal.
Then, click on the Control group tab - here you’ll see the number of customers who didn’t receive the message (because they were in the control group), and the percentage of those customers who completed the goal.
Best practices for improving your messages’ impact
You should set a time period for reviewing the impact of your control group experiment. Here are some tips for reviewing and optimizing your messages:
1. Highlight your most impactful messages
You should highlight any impactful messages. For example, if a higher percentage of users who received the message are hitting your goal compared to the percentage of users in your control group, you can likely attribute the lift in your goal’s performance to your message itself.
Once you know your message is a success, you can go ahead and:
Showcase/report on these messages to upper management to prove the impact of your work.
Identify patterns in what makes a truly impactful message and xxx.
2. Flag the least impactful messages
You should also flag any messages that don’t appear to be impactful. For example, if the same percentage of users in the control group hit your goal compared to the percentage of users who got the message, it might not be the message that’s causing your customers to take action.
We recommend, first making changes to the message - for example, you can tweak the copy or the call to action to see if your messages’ performance improves.
If your message still isn’t bringing the results you want, you should consider replacing it with a more impactful message. Your message real estate is precious, and you should only send the most impactful and helpful messages to your customers.
Can I delete a control group I’ve set up?
Yes, but there two things to note:
If you delete a control group, your message stats will be deleted.
And the users in the control group will not receive the message.
Important: Once you delete a control group, it's not possible to restore it.
Need more help? Get support from our Community Forum
Find answers and get help from Intercom Support and Community Experts