One of the issues that has been a bone of contention for as long as web sites have been created (and indeed in software interfaces for years before that) is deciding the best layout, style and functional precedence for your site/app.
It’s not easy because so often is is a case of personal preferences and opinion. We live with it because that’s the way it is, and it’s true to say that I have worked with creatives from Online Magic, Agency.com and latterly with IBM Interactive who are truly able to express with confidence an opinion that is consistent, effective, and (here’s the hardest part) interprets the requirements with a sense of style.
I read this article in MIT Technology Review about A/B testing, and had listened to a lecture in the Mobile Academy on the same subject, as well as listened to people around me.
A/B testing in it’s simplest form says lets try two or more options with the users, and measure which one has a measurable upside for a particular score or action. Will this layout/wording mean more people enter their details. Will this colour scheme generate more pull through ? etc etc.
A/B testing is seductive, it suggests I can look to replace opinion and preference with statistical facts. however this is not really true.
You still need someone with opinion, style, and a decent understanding of human’s motivations to come up with the options. Don’t blindly just try different tones of voice, colours or layouts. This will get you no where.
You need to be able to correlate the changes to the measured results. So get help in understanding the potentials, and then focus on the most meaningful. I don;t have space here to lay out a detailed example, but talk to your UX people they will know.
Also don’t think you can just follow the numbers, they can lead you down a false path, or lead you to be a “dedicated follower of fashion” which for many sites would not be wise. Also with some changes the results can be random, for example no one reads the line of text your changing, hence the assertion that you have to be able to correlate the change to the measurement, and you must be prepared fora false result.
Moreover you need to be able to persuade the brand owners that it is ok to have multiple versions of your app/site being given to users, and when the changes are not so subtle, then this is a hard argument to win. Similarly with the product owner, in the end they should make the call, so they must support the concept.
Finally remember you cannot rush these things, and indeed it may be best to be looking at this when you’re in the very early stages, when prototyping, don’t fall into the trap of trying to be perfect first time, and use variances to be testing from the get go.
All Things Mobile 
Posted on February 6, 2014
0