How To Test And Release New Features To Your Mobile App

How To Test And Release New Features To Your Mobile App

The Problem

Mobile apps never refuse new features.  But often people refuse to use them. Whenever you're adding a significant new feature, test its impact first before general release, and then tell the right users about them in the right context.

Don’t Design In The Dark

Don’t Design In The Dark

Swrve’s A/B testing functionality enables you to test specific aspects of your app by comparing how a section of your app’s user base responds to variants of in-app experience.

Run experiments on any aspect your app experience - buttons, pricing, purchasable item, configuration settings, or app UX.

A Case Study: An Airline Wanted To Drive Ancillary Revenues

In this case, an Airline wanted to drive ancillary revenues through offering hotel rooms for the duration of their passengers visit to a destination.

It’s a common use case on the web, but in mobile, given the reduced time and real-estate, it is a lot more challenging to implement.

Set up an A/B Test to know the impact of the new product feature

Set Up An A/B Test To Know The Impact Of The New Product Feature

In this blueprint, we outline how an Airline tested a new feature cross-selling hotel rooms for the duration of travellers stay.

Under this test, 5% of passengers who book a flight are served a card recommending hotels in their destination. The “Hotel Booking” card displays immediately below flight details.

The remaining 95% of passengers who book are unaffected. This experiment lets us know both the business impact of this proposed new feature, and what needs to be improved before general release.

Announcing New Features In-App

Announcing New Features In-App

Introducing new features is useless unless the right people know about them. Decide where and when to tell users about your feature.

Here’s how Snapchat, Facebook, and Instagram announce new features in-app.

Announcing A New Feature In The Context Of Real-Time Use

Announcing A New Feature In The Context Of Real-Time Use

Instagram uses in-app message in the context of use. In this example, a user is uploading a photo, but have never used the ‘combine your photos’ feature.

Instagram tell them just then. You can easily do this in Swrve; message on the precise page, and at the perfect moment - timed to the second.

Message The Right Customers Outside The App About Your New Feature

Message The Right Customers Outside The App About Your New Feature

Don’t broadcast. Be personal. If people in your control group have yet to use your new feature, message them.

Think about when and how to do so. In our Airline example, a personalized push notification delivered the next day to the passenger would likely be ideal. This is easy to set up and optimize in Swrve.

If you’ve connected your in-house email system to Swrve, a personalized ‘book your Hong Kong’ email will be delivered to the passenger.

Get Feedback From Those Users Who Just Used Your New Feature.  Especially Those Who Failed

Get Feedback From Users Who Just Used Your New Feature - Especially Those Who Failed

Swrve knows the outcome for each passenger who used the new Hotel Purchase feature. Did they convert? Did they abandon? Use this to follow up personally with each.

For passengers who converted, send a survey to find out what they liked, and how to improve. For passengers who abandoned, find out why. Normally, the best insights are from customers who declined to purchase.

Triggered email is perfect for this function. This triggering comes directly from your mobile app — “Hotel Purchase Abandoned” — direct to, and then through, your in-house email systems. Again think about timing - it may be best to delay a day before sending.

Know Your Goals And Track The Impact Of Your New Feature

Know Your Goals And Track The Impact Of Your New Feature

Use Swrve analytics to know how your experiments are performing and what you need to change.

Swrve enables you not only to track KPIs but also how A/B tests, changes to user experience, and in-app marketing campaigns affect the numbers that matter.

  • Is ARPU increasing?
  • Are people spending more time in-app?
  • Do people return more frequently?
  • Is churn up or down?
Beamly Case Study

Beamly Case Study

​One Swrve customer, Beamly, tested their traditional top banner navigation against a newer, and trendier sidedrawer or ‘hamburger’ navigation.

The results were surprising, with the traditional top banner navigation delivering 58% greater engagement over the more fashionable hamburger navigation.

Want to find out more?

Sign up for more Swrve Blueprints

Subscribe below and we will keep you updated with new additions to our Blueprints Series delivered straight to your inbox.