Booking once again impressed us with its stories at Emerce Conversion.

Two speakers, an employee, Jorden Lentze, and a former employee, Erin Weigel, told us all about Booking’s success in CRO (Conversion Optimization). Our question after this presentation was: what can we, as ‘regular’ companies, learn from Booking? And can their methods be implemented by other companies?

Booking has made a true art of their experimentation program:

• They run about 800 experiments a day.
• Each employee can ‘turn on’ those experiments.
• That means that all 2200 colleagues can set up an experiment.
• All under the inspiring leadership of a Centre of Excellence.

Booking employee Jorden Lentze explained in his presentation: “Everybody is an experimenter at Booking. It is at our core. It is part of product development. It is the way we work.”

What is the reason Booking tests extensively?

Why is Booking running so many experiments? It is the same reason Netflix and Amazon do. These are high-traffic websites with huge product teams which work daily on improving their piece of the website.

We all know that the smallest change can have the biggest impact on these types of sites, due to the large amount of traffic and interaction. This means you must validate as many changes as possible to avoid loss. And this only works if you get everyone involved. Every employee, every product team.

Booking has gathered this loss prevention into the vision of not hurting customers. Customer friendliness is paramount at Booking. Change is allowed, but it should not hurt the user, and whether something hurts or not can only be proven through an experiment.

The test tactic: quantity over quality

Of course, Booking tests to improve, but they test to avoid loss. Research is allowed, but it is not the most important thing, because Booking is all about speed. They prefer 9 ‘losing’ experiments (which will not be implemented) over 1 losing implementation no one validates. Prevention of loss is more important than the percentage of wins. Jorden explained that they call this phenomenon ‘Risk Mitigation’.

How is Booking able to run so many experiments?

In-house tool
A self-developed A/B testing tool helps the staff because this avoids complexity in setting up the experiments. It also ensures that the threshold for setting up an experiment remains low. There is, of course, another reason Booking advocates an in-house tool. It prevents dependence on external tools, and an in-house tool is much cheaper eventually, especially when used intensively.

Teams have the same composition
There is another striking aspect to Booking’s organization. Every project team is structured the same and has the exact same roles within each team.

The reason is simple. As such, the team is super flexible. It does not matter whether you are working for checkout in China or renting out apartments in Europe. Each team can easily be assigned to a project that needs help. The topic you work on may be different, the process remains the same. It makes the teams extremely flexible.

Booking does not have dedicated CRO specialists
CRO specialists? Booking does not have any. It does have designers, researchers, analysts, and developers. You work as a team and never on your own. Therefore, there are no delays because ‘CRO specialist X’s week is already full’.

And whenever there is a larger issue and more resources are needed to solve this, one or more teams are easily added. Because all teams work the same, re-assignments are fixed in a heartbeat.

Central help is facilitated
Booking has a Centre of Excellence, where they facilitate teams or individuals if needed. From their ‘cockpit,’ they can see which teams are underperforming or need help in other ways. The Centre of Excellence helps these teams with training and other forms of support to get better.

The above features of Booking have since been implemented by many followers in the field.

What makes Booking different?

Former designer, Erin Weigel, tells her story. With a smile on her face, she explained that she had once pulled down Booking’s entire mobile journey. Fortunately, with help from colleagues (who were all informed because the entire dashboard was on red alert), it was quickly fixed. Laughing aloud, she added that this had made her immortal at Booking. To this day, former colleagues are making fun of her for doing this. But she had learned from this and would not make such a mistake again.

Weigel’s message was the greatest insight of the day. As companies, we are often extremely strict about making mistakes. Weigel made it clear that a f*ckup is only annoying if you do not have a process in place to quickly resolve it. If you do have that taken care of, making mistakes is not a big deal at all. That is why no one was fired or demoted after this mistake. They learned from it, using it as an example, and moved on.

Do you want everyone to experiment eventually? Then it starts with letting go and allowing mistakes to be made. The other organizational issues will follow naturally.

The big question however is: how many companies dare to implement this way of working? We believe that a culture of experimentation starts with the participation of every employee who is allowed to make mistakes and learn from these mistakes.

Summary: The Conditions for a Culture of Experimentation

• It starts with your objective. Are you convinced that ‘mass validation’ pays off eventually? Convince your employees of the benefits of experimentation.
• If you allow everyone to be an experimenter, make sure there is no penalty for making mistakes. Let go and create a safe culture where everyone can experiment.
• Still, make sure you have a flexible workaround to quickly resolve potential mistakes.
• Put your teams and processes together in the same way so you can easily put teams on different projects, and everyone can easily pitch in when needed.
• Experimentation should be super easy. Make sure your tool is easy to use so everyone can work with it.
• Have a Centre of Excellence. You should have dedicated helpers available to ensure that colleagues keep on learning, and you can act quickly when help is needed.