Prerequisites for an experimentation culture - Center of Excellence (4/4)

This is the last of four articles.

Companies often find company culture the biggest obstacle when trying to scale up experimentation. Shared behaviors, beliefs, and values can make getting into an experimentation culture difficult or impossible.

This is the final of four articles covering prerequisites to get to an experimentation culture. 

The first three articles covered the following:

• Mindset: Three mindsets are essential to get to an experimentation culture; an experimentation mindset, a learning mindset, and a growth mindset 
• Organizational structure: In the ideal situation, cross-functional teams work together on the same goals, where information and resources are freely shared between the teams. 
• Leadership: When experimentation is fully democratized, anyone can set up an experiment and make decisions based on the outcomes without approval from higher management. 

This final article will cover the center of excellence.

Organization of experimentation personnel

Once management recognizes the value of experimentation for the entire organization and commits to scaling experimentation capability, management can organize its experimentation personnel in one of three ways. According to Thomke in his book ‘Experimentation Works’ these are: A centralized model, a decentralized model, and a center of excellence model.

Centralized model

In a centralized model, a team of specialists run experiments for the entire company. Of course, the team can involve other colleagues, for instance, by hosting brainstorming sessions. However, the centralized team manages the whole execution and resourcing of experimentation.

A centralized team can focus on long-term projects, improving the company's experimentation. Projects can include building an experimentation tool, developing advanced algorithms, data quality, documentation, and automation. The team also acts as a central point of contact.

The main disadvantage is that business units may have different priorities, which could lead to conflicts over the allocation of resources and costs. They could also resist experimentation, making it impossible for the team to do their work.

The centralized team might feel out of touch with the business units, be less aligned with the goals, and lack specific domain expertise.

In my experience: I have seen this model work for smaller organizations. In such organizations, communication is more informal, there is little hierarchy, and there are only a few business units. However, to get the most out of experimentation, the business units should eventually start experimenting themselves, resulting in a hybrid structure.

Decentralized model

In a decentralized model, different CRO specialists or leads operate in different teams. Each specialist is responsible for the experiments that happen in their business unit. There is no centralized team.

Compared to the centralized model, the primary benefit is that specialists become experts in their domains. Also, they are more aligned with the domain goals and more connected with the team members in their business units.

A major disadvantage is often a lack of information sharing between CRO specialists and no central responsibility for long-term topics such as tooling, documentation, and automation. Besides inefficiency, this also limits learning.

Center of Excellence model

Finally, there is a Center of Excellence model. In this situation, some experimentation specialists operate in a centralized function, and others within the different business units.

The Center of Excellence supports the specialists in the business units, works on long-term projects to improve the company's experimentation, and is responsible for tooling and data quality. In addition, the specialists in the business units work on the daily operations regarding experimentation.

The Center of Excellence model combines the advantages of centralized and decentralized models.

The Center of Excellence can focus on long-term projects and is a central point of contact. It can also promote experimentation and strengthen the experimentation culture throughout the organization by hosting workshops, presentations, and other initiatives.

The specialists in the business units become experts in their domains. They are more aligned with the domain goals and more connected with the team members in their business units.

There could be a need for more clarity about what the Center of Excellence owns and what the business units own. For example, who is responsible for hiring specialists, and who pays for tooling? 

However, based on my experience and many papers and articles, the CoE model is most successful in getting to a culture of experimentation and most efficient in the velocity and quality of experimentation.

Important tasks for the CoE

The Center of Excellence should be responsible for the velocity and quality of experimentation in the organization. This includes the experimentation tool, stats engine, data quality, automation, documentation, and other tools the specialists use.

However, besides stuff, the CoE has seven important tasks to motivate and enable colleagues to run as many experiments in the safest way possible, with high quality.

1. Make experimentation easy and accessible

The most important task is to make experimentation as easy as possible. When something is easy to do, it is more likely to stick and become a habit.

There are several ways the CoE can accomplish this, according to James Clear in his fantastic book Atomic Habits:

• Start with repetition, not perfection: First, have people undergo the experimentation process several times before improving the quality
• Reduce friction
• Standardize the process
• Start small

2. Educate

The CoE can make experimentation as easy as possible and increase its quality through education. 

Tailor-made education is essential to get new teams on board and have the advanced experimentation teams keep their knowledge up to date and become even more efficient. Training could also be part of the onboarding process of new employees in the company.

Workshops, manuals, coaching, and having an expert collaborate with a team that is just starting could work very well and is crucial for testing the correct hypotheses and trustworthy experiments.

3. Bring people together

The CoE should bring experimentation specialists and enthusiasts together. By creating a community, people can motivate, support, and learn from each other and facilitate organizational change together.

Bring people together through recurring meetings, brainstorming sessions, Friday meetings, visit conferences together, chat channels, or work in the same room.

4. Provide safety

Psychological safety benefits organizations and teams in many different ways. Psychological safety is showing and employing oneself without fear of negative consequences of self-image, status, or career (Kahn 1990, p. 708).

Specifically for experimentation: People will go against the status quo (e.g., come up with experiments that test new ideas) if they feel psychologically safe. They will also be more successful with experimentation, learn more from failed experiments, and be more innovative (Detert & Edmondson, 2006 & Morrison, 2011).

5. Provide efficacy

People will not contribute to experimentation if they don't believe the results will make a difference (Morrison, 2011). The Center of Excellence should ensure experimentation results are accepted throughout the company. 

Otherwise, people will stop coming up with ideas and experiments.

As the CoE often lacks the mandate, providing efficacy requires teaching and convincing stakeholders and higher management.

6. Give sufficient autonomy

People want to stay in control and are not a big fan of being told what to do. People don’t like to be managed and want to follow their own direction (Slemp et al., 2018). By allowing people to make relevant decisions independently, the CoE promotes intrinsic motivation to experiment.

7. Stimulate a culture of experimentation

Lastly, the CoE should stimulate a culture of experimentation throughout the organization and be the go-to team for (higher) management.

The team can do so by getting management buy-in, have a clear vision, write a strategic plan, apply change management, and motivate colleagues through the tasks mentioned above.

CoE as an enabler for experimentation

The CoE model is best to successfully get organizations to a culture of experimentation. Members of the CoE should make experimentation as easy as possible, allowing every employee to run experiments and make decisions based on its outcome autonomously.

Image credit: https://www.wbcsd.org/Programs/Food-and-Nature/Enabler-Service-Projects