As marketers, we’re always trying to prove and improve our impact. With every campaign we launch, audience we target, and message we send, we’re looking for positive results we can be excited about sharing with management and our teams.

But with so many factors at play, how do you measure the real impact of your campaigns? And how do you know if your efforts alone are moving the needle or if something else is influencing results?

You could launch a campaign to your user base, measure the performance, then go back in time and choose not to send the campaign and compare the results. The difference in the performance between these two scenarios would be the real impact of your campaign.

Thankfully, our solution is much simpler than time travel.

Introducing Campaign Holdouts

When you have multiple campaigns running for onboarding, converting, and retaining users, it’s difficult to isolate and measure the effectiveness of each campaign independently. For example, was the spike in online transactions today due to the push notification we sent, or were those users influenced elsewhere and would have purchased without receiving the message?

To find out, use Swrve’s new Campaign Holdouts to apply a control group to your campaign. Exclude this set of users from your campaign and then compare key metrics between the users exposed to the campaign versus those who are not. You’ll now have a more precise representation of campaign effectiveness and understand the true value of your campaign.

Note: campaign holdouts are different from holdout experiments. Holdout experiments exclude a percentage of new users from receiving all campaigns. For more information, see Holdout experiments.

Campaign Holdouts at a Glance

Apply a percentage holdback. Use the control group settings to customize the percentage of users you want to hold back from receiving the campaign. For example, you might choose to apply a larger control group to a category of campaigns your team is rolling out for the first time. For significant results, we recommend having a control and treatment size of at least 1000 users each.

Analyze the campaign holdout results. The holdout report displays the campaign holdout details and platform-based holdout metrics for your campaign, if relevant. It also shows Swrve’s default tracking metrics and results for any custom goals you selected when creating the campaign.

Compare metrics between groups. Attribute tangible value to your campaigns by comparing the control group’s activity with that of the target group. Make better marketing decisions as you gain insights such as: 

• Average revenue per user: Did the treatment group spend more than the control group during the campaign period?
• Time in app: Did users spend more time in the app when exposed to the campaign than those who were not?
• Number of sessions: Were the number of sessions per user higher among campaign recipients?

Campaign holdouts are now available as part of our Early Access program. Contact your CSM to enable this feature for your app, or visit our documentation to learn more.

We hope you enjoy this new product feature. If you have any comments or want to share what else you’d like to see us working on, we would love to hear from you. Contact our support team.

Keep an eye on our blog for more exciting product updates and releases throughout the year, or subscribe to our newsletter to get all the latest news straight to your inbox.