Today I’d like to talk a little about in-app surveys. Targeted relevant surveys are an invaluable source of insight on how your users perceive your app, the functionality it provides or needs, and whether it solves the problem you designed it for. Every product worth its salt has a long list of features on a roadmap waiting to be built. The hard part is prioritising those features to best meet the needs and expectations of your users. Surveys are often used to help this process.

On a website, with the full-screen real estate available, you have the space and you hope the attention span, to ask a few questions about how the user feels about your product and where they would like to see it go. However, on a mobile device you don’t have the luxury of that space, nor do you have the luxury of their attention. Your user is likely to be on the move, whether in mind or body. They’re waiting for the kettle to boil, just about to get the bus, or waiting for the ad break to finish on the TV.  So how do we keep the surveys concise and relevant, and yet harness the most informative results?

Recently I was introduced to the Kano Survey for measuring customer satisfaction. It’s a deceptively simple, yet powerful method of ranking your product roadmap according to how users will perceive those features. The Kano Survey focuses on asking two questions on the perception of the presence or absence of each feature.

This is best explained with an example. Let’s say you’re considering introducing social logins to your app so your users can share their status on Facebook. You’d like to get a feel for how this will impact user satisfaction within your app. The first question is something like:

If you could share your status on Facebook, how would you feel?

The important word is “feel”. The options need to be carefully worded to suit this word:

  • I like it
  • I expect it
  • I am neutral
  • I can tolerate it
  • I dislike it

These options are not a numeric scale from 1 to 5. They reflect different emotional responses to the idea of that feature being available in your app. Isn’t that all we need? Well, this is where the Kano Survey comes in to its own. The second question asks the opposite:

If you could not share your status on Facebook, how would you feel?

Notice the difference? We’re now asking the user how they’d feel if we did NOT provide the feature. And the answer options? The very same as the first time question. Huh? Let me explain. So if a user says they’d like to be able to share on social networks, but are neutral if they can’t, then this is a feature that will excite users with it’s presence but is irrelevant to them with its absence. However, if they only expect the feature to be present, but dislike the fact if it was absent, then this suggests the feature is a must-have, without necessarily winning any favours for having it.

There are many ways of analysing these results from many users, but they all distill down to categorising the paired responses according to what is called the Kano Model. Each response pair is mapped using the following table, based on the “positive” and “negative” questions, to one of 5 categories.

Attractive - These features trigger feelings of satisfaction when present, but users are not dissatisfied if the feature is absent. These features are unexpected and invoke surprise. These generally reflect the USP of your product.Here in lies the beauty of a Kano Survey. It allows you to categorise your features into the following types:

  • 1-Dimensional - These features drive satisfaction when present and dissatisfaction when absent. The sophistication of these features generally provides a lever with which to increase customer satisfaction and provide small wins over your competition.
  • Must-Have - These features are expected by users, as a default. They reflect the baseline, minimal product you should have to avoid dissatisfied users, never mind gaining satisfaction. These features are needed in your app, but don’t waste time trying to make them too fancy.
  • Unimportant - These features do not invoke good or bad feelings in users, and so are essentially irrelevant to the success or failure of your app. Any features that fall in to this category should be avoided.
  • Undesired - These features are interesting, albeit thankfully rare. The presence of these features is likely to frustrate users and lead to dissatisfaction. For example, this could happen if a proposed feature is deemed to overly complicate or simplify a process that the current user base is well used to. If your survey exposes any of these, it’s worth revisiting your strategy and reasoning before spending any time on development.

In the context of mobile surveys, Swrve would recommend targeting a simple Kano survey addressing one feature to a random subset of users. We’d recommend the target audience is tuned to reflect the feature being suggested. Different features should be surveyed against different subsets of that target audience. The outcome of these surveys will greatly assist in categorising your features so each major update to your app includes must-have, one-dimensional and attractive features.