In this post, I want to look at how CRO can excel and flourish in larger organizations.

The idea of experimentation

Through our experiences and our knowledge, we may know what the right action is. We may know which markets to enter, how to price our products and services, and ways to acquire customers. Sometimes our decisions are spot on, and we take market share and unfortunately, sometimes we lose both time and money by making wrong decisions. 

The idea of experimentation is to help mitigate the level of risk by taking a big decision and looking at cheap ways to test it. By testing, we can find out what resonates with our audience i.e. moving our audience to a desired action e.g. signing up for an email, adding a product to a cart, etc. 

As an example, rather than investing in a new product range without understanding the market, we can simply create a landing page and collect email addresses for those who are interested in the product range. This way we can gauge interest in the product. 

Experimenting gives insight and direction that can be easily missed if we decide what to do, without knowing how the market will react. 

CRO functioning in an organization

In some organisations, the marketing or e-commerce manager is responsible for doing CRO, whether it is to optimise a marketing campaign or make a change to the website. Here CRO is considered as an additional responsibility. 

In other organisations where CRO is seen as a core function and more a team effort, there are two ways a team can be organised. Centralised and decentralised.

Centralised CRO function

For many organisations, if you look at the org chart you will see that staff are organised into functions. They will be a team that looks after e-commerce, another for acquisition, another for retention and another as an example of business intelligence. Each team will have their speciality and they will serve the organisation as a whole. 

The same can be applied to CRO. A centralised team that often works under a service model like an internal digital agency. Their focus is on optimising websites, mobile/web applications and campaigns across the organisation. 

The benefits of a centralised team

  • Specialisation as the team will all focus on one thing
  • A centralised team is more likely to document and be systematic in their approach. Such an approach can be scaled. 
  • Lessons learnt from one part of the organisation can be applied to other parts of the organisation
  • Efficiency gains by comparing projects across the organisation, the ones with the highest potential return on investment will be prioritised. 
  • A centralised team is more likely to gain respect and acceptance from the organisation. 

The disadvantages of a centralised team

  • Having a central team creates a distance between the business area to optimise and the team that does the work. They may not have the detailed knowledge and experience of the audience relative to someone who works within that business unit.
  • If the CRO team selects projects with the best potential return it can mean that the smaller business units or small projects are never optimised. 
  • Scaling a centralised team can lead to inefficiency in communication and their ability to make decisions. 

Decentralised CRO function

A decentralised CRO model is when different teams manage CRO for the various parts of the organisation individually. For example, they could be a team that manages CRO for a website, another for the mobile app etc.  

The benefits of a decentralised CRO function

  • By focusing on one area of the business the CRO team can become experts in that area. They understand that part of the business better than someone who oversees multiple areas of the business. 
  • Being part of a business area helps to influence the business/product roadmap more than a centralised business unit
  • A decentralised approach allows for more tests in that particular business area as prioritisation will only focus on that area of business rather than the whole business

The disadvantages of a decentralised CRO function

  • In the absence of detailed cross-departmental communication, it is easy for different teams not to share their learnings, and their KPIs do not align so their testing could work against each other
  • There could be a duplication of resources as each team needs to function independently

Framework for CRO

A CRO framework is the strategic approach that a business adopts that guides and drives its experimentation. It establishes how to prioritise experiments, setting up a test and the resources required e.g. QA for running a test. It would help by defining the steps of what should be tested:

  • How to identify what to test
  • What are the levers which can change the influence 
  • How to rank the opportunities
  • Writing a test plan

Having a framework provides the following benefits:

  • Helps create consistency across the organisation
  • Helps new starters get up to speed
  • Allow more people to create tests
  • Saves time in setting up and managing experiments
  • Frees up time to focus on strategy/direction of experimentation
  • It helps transition from tactical testing to strategic testing

Tactical vs strategic testing

Tactical experimentation treats every experiment independently from other experiments. There is no overarching strategy, and the objective of each experiment can be vastly different. If one experiment does not lead to a positive result, rather than iterate on it, a completely different experiment would be selected.  

Strategic experimentation has a predefined goal to solve a problem using different strategies. These strategies are tested by a defined hypothesis. All experiments are therefore connected and iterative as we develop further insight into how to achieve our goals and adjust our future experiments. 

A successful CRO team 

it would be valuable to consider what characteristics help make a CRO team successful within a larger organisation. Larger organisations function differently from start-ups/smaller companies and different from digital agencies. Reporting lines are longer, time to execute may require navigating through red tape and resources may be abundant but require business cases to become available. 

We will first begin by looking at the characteristics of leadership. 

Characteristics of leadership

  • Able to navigate through the org chart, knowing who to talk to to get things done
  • Provide visibility to the wider organisation and an ability to build a business case
  • Able to influence the organisation to become more Agile
  • Embed experimentation and the prioritisation principles within the organisation
  • Lead by example. I would highly recommend reading the book Leaders Eat Last by Simon Sinek 
  • Lead not manage
  • Promote a culture of collaboration and share results far and wide
  • Create a safe space where failure is considered a part of learning and success is celebrated
  • Hold people accountable but give them the space to figure things out

What skills does a CRO team require?

Whether a CRO team is centralised or decentralised the team members collectively require several skills to deliver effectively. We can classify those skills into two broad categories. Technical skills and soft skills. 

Technical skills

Analytics/Statistics – An understanding of web/mobile/product analytics so they can interpret data, derive conclusions, make data-driven decisions to identify what to focus on, set the measures in an experiment and analyse the results. 

UX/UI – The ability to design the functionality (UX) of a page/flow optimized for user experience. Develop user-friendly interfaces (UI) and conduct user research.

Design – The ability to illustrate, develop branding, create visually appealing designs to attract attention etc. 

Psychology – It is essential to understand how humans think. What makes one person buy a product over another? Or what information is required on a product page for different personas? 

Copywriting – The ability to effectively communicate in written prose. Create a persuasive and compelling copy to drive conversions.

Front End Development – An understanding of web technologies so they can optimise pages and build A/B tests

Digital Marketer – launch marketing campaigns and landing pages and manage the optimisation of those campaigns and pages

Soft skills

Embrace a Growth Mindset – see challenges and defeat as an opportunity to learn and grow. Taking the learnings from an experiment whether successful or not as an opportunity to further optimise and improve.

Willing to take risks and manage failure – experimentation requires taking risks, but it also requires you to do your best to mitigate worse-case scenarios. Understanding the balance between having users to test and still hitting your revenue/user numbers is a skill.

Creativity – Finding solutions, thinking out of the box, and being innovative are all essential.

Collaborate – In any team especially a multi-functional team the team members must work well with each other. This is something that does not happen overnight, and it is something that needs to be continuously worked on. 

Ego in check – Everyone has an ego, it is part of the makeup of the human being. An ego out of control can destroy the fabric of what holds the team together and derail a test-and-learn strategy. 

Decision making – Following on from ego, decisions mustn’t be based upon whims and desires to help keep egos in check. Decision-making should be evidence-based using data analysis and the results from experimentation.

One team – team members should be empowered to propose, and priorities A/B tests to validate their hypothesis and find solutions to problems. Decision-making is shared amongst the team and not led top-down.

Communicate and share – team members introverted or extroverted should be comfortable communicating within the team and to the larger organisation. 

Forever learning – learning best practices, keeping to to date with the latest CRO theories and ideas helps the team grow and develop.

Photo by Antonio Janeski on Unsplash