Most of us working in the digital world are already more than familiar with the benefits of A/B testing. Time after time, we see demonstrations of the way in which real user data triumphs over our own 'best guess' or even informed opinion (although let's be honest, the two are often interchangeable). Although the mobile app world has been a little slower to the party, there can't be any doubt that as in the broader online world, we'll soon see A/B testing as an absolutely standard part of the development process.

But that raises another question. Being all for testing is one thing, but it's fair to ask WHAT we should be testing. Of course there is a short answer to that question: whenever you ask yourself the question “what if?” - test. But the truth is that although there's a certain amount of truth in that answer, it doesn't help the organization with limited resources that wants to identify and target those aspects of the app that can really deliver ROI. There are an almost limitless number of things that CAN be tested - and they cover the entire range of user experience. They certainly are not limited to the size and colors of buttons.

So how DO you choose what to test? Here's 5 tips to help you do just that:

1. Let Your Data Guide You

Without analytics - and a clear understanding of where your difficulties and opportunities arise - you'll find it difficult to know where to begin. That's why standalone A/B testing platforms can be frustrating. The way to find the opportunities for testing that will really make a difference is in data about the existing situation that highlights them. To give just two obvious examples - if at a certain stage in your tutorial many users are leaving (as identified in a funnel analysis), it might be worth testing the user experience at that point. And if a large number of users are 'bouncing' after hitting an in-app store, perhaps it's time to test every aspect of that initial store front?

2. Find Large Audiences

10% of a large number is more valuable than 10% of a small number. Put simply - don't waste your time optimizing processes only used by a small minority of your users, even if in headline terms you can see large % increases in metrics within that test. In practice that means focusing on the things that a lot of people do. And in turn that probably means the initial experience (which by definition everybody does) and the most common tasks your app is used for. After all, if you can't get your core functionality delivering, you've got a real problem.

3. Get Close To The Money

Some decisions are more important than others. In the mobile app business, or at least large parts of it, getting users over the line to a first purchase is an absolutely critical juncture in the user experience. It's a moment that many marketing campaigns are built around, and a huge amount of app design goes into making happen. With that in mind, the specific steps running up to that moment are probably worth testing to destruction. Remember - that can be modeled using data as above - and the results used to inform exactly what point in that process is most worthy of our attention.

4. When The Alternative Is Sitting Around A Table - Test

We suggested above that the answer “whenever you ask 'what if?'” was a little too trite as a way to determine what to A/B test. But if you consider a problem or issue to be important enough to sit around a table and argue until a decision is reached - you are looking at a candidate for testing. There are a number of examples I could offer, but one that we are all familiar with is the concept of a 'guide' character in a mobile game. The usual process might be to design several alternatives and then choose one internally. Why not test? It is simple enough to incorporate each variant and establish how they play with users - in fact it's probably simpler, safer and less exhausting than generating a lot of hot air and precious little insight…

5. Small Decisions That Have Big Impacts Down The Line

Some apparently trivial decisions can have major consequences. One we discussed in our last webinar was the decision around when to request opt-in for push notifications. Too early, and you'll intimidate your audience and see a low rate of opt-in. Too late and half of them have left already! And the number of opt-ins for push notifications, whilst of no real significance in itself, has a major impact when it comes to the effectiveness of your campaigns in future. Always look for those moments of decision for your customers that will define your future relationship - and how effectively you will be able to talk to them in future. Whether or not to enforce registration falls into the same category.