CRO - Is your organisation ripe for conversion optimisation?

The most well-known CRO activity is probably the A/B test. Version A is pitted against version B – the one with the best results wins. It sounds simple, but the challenge lies in making use of the results from the experiment.

"To invent you have to experiment. And if you know in advance it's going to work, it's not an experiment." Jeff Bezos, founder of Amazon

We spoke to Cecilia Jeppsson, Head of Insights at Bombayworks. She likes the term "experiment" and has a few questions for those who want to invest in CRO.

1. Is the problem defined?

– The most important thing is that you have done your research, understand the business and know what problems it faces. What you think is a conversion problem may turn out to be an acquisition problem, says Cecilia Jeppsson.

2. Are the right skills available?

A/B testing requires input from creators, analysts and developers. A good team can consist of the following roles:

• Creators A designer with UX expertise develops new design proposals, a copywriter adjusts the messages. The exact division of roles depends on the nature of the test.

• Developers A developer may be needed for the implementation of design and tools. A CRO specialist can usually handle minor changes themselves.

• CRO specialist An expert in conversion optimization sees testing opportunities and interprets test results. More specifically, the specialist develops hypotheses and follows up on the final results.

– Skills such as acceptance testers, data scientists and project managers may also be relevant, it depends on the organization's maturity level, says Cecilia Jeppsson.

3. Is the tool available?

Once you have inventoried your needs, challenges and skills, the tool can be implemented. It is important that this is done correctly to avoid disrupting the user experience. In the case of A/B testing, incorrect implementation can result in "flickering" (a phenomenon where the "wrong" version becomes temporarily visible). The choice of tool often becomes a budget issue, but there are variants for both large and small companies.

– I have used slightly different tools for A/B testing, but so far, my favourite is Optimizely. It is often aimed at slightly larger organizations, so if you want a cheaper alternative, you can use Google Optimize," says Cecilia Jeppsson.

4. Are the right processes in place?

A common framework for A/B testing is based on four areas:

• Ideation This is where the hypothesis is primarily developed. A good one is based on qualitative or quantitative insights.

• Experimentation This is where the method is determined, followed by actual implementation, testing and follow-up. The experiment is followed up continuously and stopped no earlier than 2 weeks after the start.

• Recommendation After the test is completed, the results are reviewed, which leads to one or more recommendations.

• Action In the last step, the recommendations are translated into action, often in the form of implementations, iterations or a return to the ideation stage.

5. Is the right mentality in place?

Many organizations consider themselves to be data-driven, but an alarming number only place importance on data that supports their own agenda. A conversion optimizer proceeds scientifically, armed with a hypothesis based on a problem that needs to be solved. A data-driven organization learns from all kinds of results, something that can lead to insights and growth. An organization that has this mindset at all levels has achieved full maturity in CRO.

Successful optimization programs are more than just a (correctly implemented) tool. It is an almost scientific process that requires a steady stream of data, a team of skills and an analytically aware organization. Are you ready to take on the challenge?

Publication: Byråvärlden, February 2020