Generating a theory
We have a very strong relationship with our client, they have a lot of trust in our team. This has lead to a collaborative process centered on continual AB testing. AB testing or variant testing is pretty simple at it’s core. Using analytics or browsing the site we notice areas that could be improved whether it’s the copywriting, the design, imagery, or user controls. We work collaboratively with our client to develop hypotheses that identity problems and generate various ideas to improve them.

Creating tests
With a hypothesis in mind we start developing tests. In other companies I’ve worked, designers were expected to develop the one perfect, bullet proof idea that solved any problem on the first attempt. This idea solved the problem “perfectly” and was the end all answer. Our team at MxM works completely differently.

We have a theory of what might be hindering the user and then develop two or three solutions to test, each idea being intentionally different from the others. The goal of a design in this scenario shifts, from being right to be being wrong faster. If a revision doesn’t generate positive or negative results then we’ve not learned enough to make a decision to improve the site.

Being wrong faster
From a design standpoint you’re no longer fixated on making an idea fool proof. Instead you’re taking a nugget of an idea and designing many different ways that concept could be executed. For example, take a control where you’re picking options before checking out. We designed a drop down, a grid you fill in from a secondary grid, and a big carousel you move through one at a time to pick from. Our team launches these ideas and we watch what happened. The safest idea, the drop down, won by clear margin. But had we chosen to move forward with the “cooler” grid picker we should have unknowingly handicapped our site, ultimately affecting its performance.

That winning concept becomes the new design on the site and we then start thinking of the next round of ideas to experiment with. Sometimes this is several new versions of what we just worked on to further refine it or we explore an entirely new problem.

Embracing testing
For a designer this process is somewhat unsettling at first. You’re accustomed to exploring several ideas but eventually picking a single route to refine. Designing for testing means you’re giving up some control. You’re no longer self editing ahead of time, but letting real users pick the idea. This feedback cycle also stretches from rounds of in-studio critiques to weeks of watching data flow in before making a decision. It can be frustrating at first, but you gain the confidence that pushes better design onto the web, and in the end it’s what we’ve always strived for.

Better results with less work
Switching to this process I’ve found myself no longer hung up on every little detail in each design I’m working on. If I’m stuck between two versions of a design, then I have an AB test lead the way. Don’t leave it up to a big team of opinionated people, leave it up to thousands of people using your design everyday online. This has cut down on inter team debate, massive design reviews, and weeks of personal iteration. Instead designs move from ideas to live within a week or two; feedback comes in a few weeks later and we do the whole cycle again.

I encourage everyone that has access to AB testing for their projects to embrace it. You’ll find yourself coming up with better designs faster. It’ll feel uncomfortable for the first month or two but soon you’ll see results in the form of a great site and it will be worth it.