In this post, I want to consider how CRO can flourish in larger organizations.

The idea of experimentation

Taking our experiences and our knowledge, we may know what the right action is. We may know which markets to enter, how to price our products or services and the different ways to acquire customers. Sometimes our decisions are spot on, and we take market share. On other occasions, we unfortunately make the wrong decision and lose both time and money. 

The idea of experimentation is to help mitigate risk by taking a big decision and looking for cheap and easy ways to test it. By testing, we can find out what resonates with our audience i.e. does it make our user take a desired action e.g. signing up for an email, adding a product to a cart, etc. 

As an example, rather than developing a new product range without knowing how well it would do, we can create a landing page and collect email addresses for those who are interested in the product range. This way we can gauge interest in the product before investing time and money. 

Experimenting gives insight and direction that can be easily missed if we decide what to do, without knowing how the market will react. 

CRO functioning in an organization

In some organisations, the marketing or e-commerce manager is responsible for doing CRO. This could be optimising a marketing campaign or making changes to the website. In this example, CRO is considered an additional responsibility. 

In other organisations, CRO is seen as a core function and more a team effort. The teams can be centralised and decentralised.

Centralised CRO function

For many organisations, if you look at the org chart you will see that staff are organised into functions. They will be a team that looks after e-commerce, one for acquisition, one for retention and another for business intelligence. Each team will have their speciality and they will serve the organisation as a whole. 

The same can be applied to CRO. A centralised team often works under a service model like an internal digital agency. Their focus would be across the full organisation’s websites, apps and campaigns. 

The benefits of a centralised team

  • Specialisation, focus and shared learning
  • A centralised team is more likely to document and be systematic in their approach. Such an approach can be scaled. 
  • Lessons learnt from one part of the organisation can be applied to other parts of the organisation
  • Efficiency gains by comparing projects across the organisation, the ones with the highest potential return on investment will be prioritised. 
  • A centralised team is more likely to gain respect and acceptance from the organisation. 

The disadvantages of a centralised team

  • Having a central team creates a distance between the business area to optimise and the business area. They may not have the detailed knowledge and experience of the audience relative to someone who works within that business unit.
  • If the CRO team selects projects with the best potential return it can mean that the smaller business units or small projects are never optimised. 
  • Scaling a centralised team can lead to inefficiency in communication and their ability to make decisions. 

Decentralised CRO function

A decentralised CRO model occurs when each business area or unit manages the CRO internally. For example, they could be a team that manages CRO for a website, another for the mobile app etc.  

The benefits of a decentralised CRO function

  • By focusing on one area of the business the CRO team can become experts in that area. They understand that part of the business better than someone who oversees multiple areas of the business. 
  • Being part of a business unit can help to influence the business/product roadmap more than a centralised business unit
  • A decentralised approach allows for more CRO activities in that particular business area as the focus will not split amongst different business units.

The disadvantages of a decentralised CRO function

  • In the absence of detailed cross-departmental communication, it is easy for different teams not to share their learnings
  • KPIs may not align across departments
  • There could be a duplication of resources as each team needs to function independently

Framework for CRO

A CRO framework is the strategic approach that a business adopts that guides and drives its experimentation. It establishes how to prioritise experiments, creating a test and the resources required e.g. what should QA test for a test goes live.

Having a framework provides the following benefits:

  • Helps to create consistency across the organisation
  • Helps new starters get up to speed
  • Allow more people to create tests
  • Saves time in setting up and managing experiments
  • Frees up time to focus on strategy/direction of experimentation
  • It helps transition from tactical testing to strategic testing

Tactical vs strategic testing

Tactical experimentation treats every experiment independently from other experiments. If there is no overarching strategy, then the objective of each experiment can be vastly different. If one experiment does not lead to a positive result, rather than iterate on it, a completely different experiment would be selected.  

Strategic experimentation has a predefined goal to solve a problem using different strategies. Strategies are then tested by a defined hypothesis. All experiments are therefore connected and iterative as we develop further insight into how to achieve our goals and adjust future experiments. 

A successful CRO team 

it would be valuable to consider what characteristics help make a CRO team successful within a larger organisation. Larger organisations function differently from start-ups/smaller companies and differently from digital agencies. Reporting lines are longer, time to execute may require navigating through red tape and resources may be abundant but require business cases to become available. 

We will first begin by looking at the characteristics of leadership. 

Characteristics of leadership

  • Able to navigate through the org chart, knowing who to talk to to get things done
  • Provide visibility to the wider organisation and an ability to build a business case
  • Able to influence the organisation to become more Agile
  • Embed experimentation and the prioritisation principles within the organisation
  • Lead by example. I would highly recommend reading the book Leaders Eat Last by Simon Sinek 
  • Lead not manage
  • Promote a culture of collaboration and share results far and wide
  • Create a safe space where failure is considered a part of learning and success is celebrated
  • Hold people accountable but give them the space to figure things out

What skills does a CRO team require?

Whether a CRO team is centralised or decentralised the team members collectively require several skills to deliver effectively. We can classify those skills into two broad categories. Technical and soft skills. 

Technical skills

Analytics/Statistics – An understanding of web/mobile/product analytics so they can interpret data, derive conclusions, set the measures in an experiment and analyse the results. 

UX/UI – The ability to design the functionality (UX) of a page/flow optimized for user experience. Develop user-friendly interfaces (UI) and conduct user research.

Design – The ability to illustrate, develop branding, create visually appealing designs to attract attention etc. 

Psychology – It is essential to understand how humans think. What makes one person buy a product over another? Or what information is required on a product page for different personas? 

Copywriting – The ability to effectively communicate in written prose. Create a persuasive and compelling copy to drive conversions.

Front End Development – An understanding of web technologies so they can optimise pages and build A/B tests

Digital Marketer – launch marketing campaigns and landing pages and manage the optimisation of those campaigns and pages

Soft skills

Embrace a Growth Mindset – see challenges and defeat as an opportunity to learn and grow. Taking the learnings from an experiment whether successful or not as an opportunity to further optimise and improve.

Willing to take risks and manage failure – experimentation requires taking risks, but it also requires you to do your best to mitigate worst-case scenarios. Understanding the balance between having users to test and still hitting your revenue/user numbers is a skill.

Creativity – Finding solutions, thinking out of the box, and being innovative are all essential.

Collaborate – In any team especially a multi-functional team the team members must work well with each other. This is something that does not happen overnight, and it is something that needs to be continuously worked on. 

Ego in check – Everyone has an ego, it is part of the makeup of the human being. An ego out of control can destroy the fabric of what holds the team together and derail a test-and-learn strategy. 

Decision making – Following on from ego, decisions mustn’t be based upon whims and desires to help keep egos in check. Decision-making should be evidence-based using data analysis and the results from experimentation.

One team – team members should be empowered to propose, and prioritise A/B tests to validate their hypothesis and find solutions to problems. Decision-making is shared amongst the team and not led top-down.

Communicate and share – team members introverted or extroverted should be comfortable communicating within the team and to the larger organisation. 

Forever learning – learning best practices, and keeping up to date with the latest CRO theories and ideas helps the team grow and develop.

Photo by Antonio Janeski on Unsplash

The journey 

I work in a software organisation that uses Scrum to lead our development teams. Outside the development team, the business unit is structured in functional silos e.g. marketing, design etc. With the success we had with using Scrum, we believed by adopting Agile principles in other areas of the business we could change the way we work for the better.  

At the time of writing this post, I have been running an Agile squad for a few months, and these are some of the learnings I have had. 

The most important things I learnt 

If I were asked what are the top three most important things I have learnt, they would be:  

  • The team – a vital component of success is in harmony amongst the team members
  • Test, measure & adapt – Find a quick and easy to test your ideas. It is better to be wrong and learn something. Then to be right, but not know why. Businesses scale when they know why.
  • Focus on an objective that is possible to achieve but impossible to master 100%

The principles we started with 

  • Change takes time so start small  
  • The squad members have responsibilities outside the squad, so do not overburden the team
  • Being a member of the squad is entirely voluntary. Build something amazing so people want to join

Where we started 

To build the squad, we needed people to cover every major business function (UX, user research, design, development and marketing) and some particular requirements for our business model. This way the team was not dependent on another’s team’s schedule and could deliver rapidly.  

We also had soft skills that we felt were essential for the team (happy to learn through iteration, able to accept change and able to work effectively in a team). 

It was essential to get the business leader’s buy-in as members of the squad would come from their team. Once we knew what we wanted, we spoke to the team leaders and explained what we were looking for. They then spoke to their teams trying to match the experience of their teams, the availability of individual members and their desire to work in an Agile squad to meet our requirements.  

The interviewing process 

Once we had a list of possible candidates it was time to speak to them. For us it was important to understand: 

  • What are their expectations? 
  • What skills do they bring to the team? 
  • Are they a good match for the team?
    • Do they welcome change?
    • Are they happy to work inside an Agile framework?
    • Are they interested in the projects we will work on? 
  • How much time can they devote to the project? 

What I learnt from this experience: 

  • Being excited does not necessarily translate to commitment to the squad. Some of the more reserved individual have a deep passion for their work. 
  • Agile is a buzzword so it is important to clarify what they understand Agile is?
  • Clarify the level of participation they are comfortable with. Do they want to lead? Are they are interested in trying things outside their core skill set? Or are they prefer to work inside their comfort zone? Do they have any major upcoming projects coming up?

Kick-off meeting  

Once we have the team it is important to have a kick-off meeting. The objective were twofold: 

  • Let everyone get to know each other 
  • Outline how the squad will work  

What our kick-off meeting looked like: 

  • Objectives – Explain the mission of the squad and how it plays into the vision of the business. 
  • An icebreaker activity – to create a space where people can open up  
  • Metrics – how will we measure success and failure 
  • Failing fast – ideas of MVP
  • First steps – what are the first round of projects we will work on 

What I learnt from the experience 

  • Build trust first 
  • Outlining initial projects helped the team adjust to working in a squad 
  • There are many smart people in the room, listen attentively

Having regular squad meetings 

As a squad we organised regular meetings so members of the team could come together to discuss

  • What are they working on?
  • Any potential roadblocks?
  • Refining what is next

What I learnt from having squad meetings:

  • Regular squad meetups are instrumental in improving communication and collaboration 
  • Ask for feedback
  • Not everyone will participate in a meeting. Some may prefer to share, comment or give feedback after the meeting.
  • Have meetings optional except the kick-off meeting respects the squad member’s time. If certain members of the squad need to attend a meeting, then it needs to be explained why they should attend.  
  • Publish meeting notes after the meetings to act as a reminder of the outcome of the meeting and for those who did not attend to understand what was discussed. 

Sharing is power – communication and virtual whiteboards 

People learn differently, they understand things differently, some are auditory, some are visual, and some are tactile learners. By sharing information openly, we achieved: 

  • Sharing early and often helps save time in the long run.  
  • If it is documented, it is easy to reference in the future. Especially when onboarding new members. 
  • Sharing information creates a culture for others to share as well.  
  • Helps remove follow-the-leader mindset and puts a value on the contributions of all members of the squad 

The ego and the need to be right 

Transitioning from a top-down approach where the highest-paid opinion counts more and to an equal playing ground can be a jarring experience. The leader becomes a servant leader rather than the centre of the team. A meeting or a project can easily be derailed when a difference of opinion arises, so it needs to be resolved before moving forward with logic and evidence and not because I am paid more than you.   

The take-home message from this experience it can be a very humbling experience. Success in Agile is about swallowing your pride and accepting that evidence/testing is more important than your opinion and experience.  

A shared brief 

In a business where teams are organised by function, it is typical for the business owner to produce a brief for a project. This brief becomes the source of truth and will be used by other teams to deliver on that project. In the Agile squad model, the cross-functional team meet to discuss a brief, this gives everyone the option to voice their opinion on the brief before it is finalised.  

What I learnt from this experience 

  • Encourage critiquing of ideas as everyone has a perspective
  • Meetings can easily get sidetracked. It is important to ensure the discussion is on point.  
  • Use virtual whiteboards to brainstorm and track conclusions 
  • Some team members will need to be encouraged to speak 
  • Ask the question how do we test? 
  • End the meeting with action items 
  • Produce a summary write-up after the meeting and share it with the team 

Prioritisation and Impact 

One of the first exercises we ran was to rank a list of projects by impact and ease of implementation. For impact, we look at estimating revenue generated and ease of implementation. We used t-shirt size to measure the level of complexity and forecasting to estimate revenue.  The ideas were then sorted into now, next and future, this gave us our pipeline.  

What I learnt from this exercise: 

  • Estimating revenue is difficult, but expert help is a lifesaver 
  • Using t-shirt sizes simplified the comparison between projects 
  • Using now, next and future not only gave us a pipeline of activities but helped to unify the team under a roadmap 

First project – the first win 

I believe it is important to take baby steps as the team is coming together. Our first project was simple to implement and likely to be successful. This gave us an early win and time to come together as a team.  

MVP (minimum viable product) 

An MVP is a concept from Lean Start-up. A proof of concept is built and tested. The results determine what action the squad can take. 

What I learnt from this experience 

  • MVPs do save time and effort in the long run but in the short run, they can create more work for the squad. 
  • A successful MVP is not a complete product. There is usually duct tape holding it together. 

Running a retrospective 

Our developers run a retrospective at the end of every two-week Sprint. In our squad, we did not have Sprints, but we still use retrospectives. The purpose of the retrospective is to take stock of what has been achieved, the good, and the bad. Thus giving us the ability to improve. 

Everyone answers three questions  

  • What should we continue to do? 
  • What should we start doing? 
  • What should we stop doing? 

Once everyone has spoken, we then get a vote on the top 3 areas of improvement.  

What we learnt from this experience 

  • Look for themes in the feedback 
  • The value of the retrospective is to act 
  • Within a week after the retrospective report back on improvements implemented

Dealing with priorities and overload 

As the squad member had other priorities it is easy to pile on the work without understanding the impact it has on the individual team members. In my experience, it is better to set a target that is based upon the ability to deliver then an unrealistic fanciful deadline on when you think it should be done by. More importantly, regular check-ins can help to maintain, build or slow down the momentum.  

If a key person cannot deliver: 

  • Do they need help to prioritise? 
  • Can we assign the work to someone else? 
  • Should we focus the team on the next top priorities while we wait for them to become available? 

Definition of done 

To end I wanted to mention the importance of the definition of done. In Scrum, the definition of done is an artefact. The Scrum guide states: 

The Definition of Done creates transparency by providing everyone with a shared understanding of what work was completed as part of the Increment. If a Product Backlog item does not meet the Definition of Done, it cannot be released or even presented at the Sprint Review. Instead, it returns to the Product Backlog for future consideration

In our squad, as we come from different specialties what we see as done can be very different. We had to work to build a shared understanding of done. As an example when something is done, it can still go through additional rounds of iterations.

Images taken from Pixabay and Unsplash

Photo by on Unsplash

Photo by Matt Benson on Unsplash

Photo by Mark König on Unsplash

Principle 8 from the Agile Manifesto

Agile processes promote sustainable development. The sponsors, developers and users should be able to maintain a constant pace indefinitely.

What is Burnout?

According to Psychology Today burnout is defined as:

Burnout is a state of emotional, mental, and often physical exhaustion brought on by prolonged or repeated stress. Though it’s most often caused by problems at work, it can also appear in other areas of life.

According to a recent Gallup study of nearly 7,500 full-time employees found that 23% of employees reported feeling burned out at work very often or always. An additional 44% reported feeling burned out sometimes.

Burnout is attributed to long hours, little downtime, and continual peer, customer, and superior surveillance. Source

Signs of Burnout

  • Chronic fatigue
  • Insomnia
  • Physical symptoms like headaches and stomach aches
  • Anger
  • Detachment from co-workers and customers
  • Extreme dissatisfaction with your work

Causes of Burnout

  • Negative work culture
  • Lack of work-life balance
  • Lack of control over the direction of your career

Why does burnout exist within an organisation?

Reasons include

  • Lack of prioritisation so everything is done at once
  • Lack of a coherent strategy and direction
  • Not distinguishing between what is urgent and what is important
  • Poor management 
  • Putting short term profit above people
  • Treating staff as a factor of production rather than a human being

What are the implications of burnout to an organisation?

Work-related stress costs British businesses over £26 billion a year in absences, sick leave and disability benefits. 

We can look at the impact of burnouts at three levels:

  • Individual cases
  • Team level
  • Organisational level


Work-related burnout can be devastating to the individual. It can take time to recover and it can leave long term mental and physical damage. For some, it may even mean changing careers or being away from work for long periods. 

Team level 

If a team or department exists within a high-stress environment then the following issues may be seen:

  • Drop-in productivity 
  • High staff absenteeism 
  • Demoralised teams 
  • Increased staff turnover
  • More mistakes are made

Organisational level 

Burnout leads to low staff morale, turnover of staff, lower product quality and ultimately impacts the bottom line. This also can negatively impact the company culture, increase the workload for all staff and could lead to legal cases as well as negative press.

Time, product features and quality

Before we look at sustained development it is important, we consider the idea of delivery.

In every project there are three factors:

  • The deadline to deliver the project
  • The product features that the project will deliver
  • The quality of the features that are built

If the deadline is fixed and it is not possible to deliver all the features in time then we may be faced with two possible scenarios:

  • Apply pressure on the team to deliver regardless of the impact it has on the team
  • Compromise quality which impacts the user and builds tech debt for the technical team

Commonly the team would be expected to work longer hours to deliver. Working longer hours and trying to squeeze everything in can impact impact quality. Compromises are made to meet the deadlines, QA may not have sufficient time to full test and any feedback may not get actioned by the development team. 

From an Agile perspective quality (the definition of done) is not compromised rather we limit the number of features that can be delivered in the timeframe given. The product roadmap then maps out the order of the features that will be developed. 

What is Sustained Development?

Sustained development is one of the principles from the Agile Manifesto. According to the Agile methodology, the pace of development is set by the development team and not by managers or external parties. The team understand their capacity so they can scale up and scale down according to their circumstance. Therefore the pace of development can be continued indefinitely because it adjusts to the particular needs at the time. 

Pace of development

One of the aspects of sustained development is the ability to say no. The team has a capacity and anything above and beyond that capacity may not be done straight away. This introduces the idea of opportunity cost. Opportunity cost is the loss of other alternatives when one alternative is chosenIf you work on one story then the cost is the lost opportunity to work on another story.

One of the features of Scrum is the team is self-organising. The development team will decide what it will work on per Sprint. The product manager can set the priorities in the product backlog and explain their rationale but ultimately the developers decide what will enter the Sprint backlog. The Sprint backlog determines the pace of the development which is agreed upon by the development team. 

If the developers are on annual leave or ill, they can adjust the number of stories they commit to in that Sprint. If the dev team are trying to implement a story in new technology in the Sprint planning session can award the story a higher Fibonacci story point because of the uncertainty of a new language. This helps to protect the well-being of the team. The opposite is also true, as the team become experienced they are able to complete my stories per Sprint. 

If during the Sprint an urgent and important story is presented to the development team, the product manager and the development team will negotiate what needs to be removed from the Sprint to focus on the urgent story. This also helps to protect the well-being of the team and ensures that only the stories with the most value are developed first. 

The final example in this section is the idea of velocity and story points. Stories points are not compared between teams and it is something that is consumed within the team. What one team may classify as 5 points another may see it is as a 13-point story. It is the team that defines the amount required to deliver a story and it is not imposed upon them by an external party. They may seek external expert advice but it is the team who ultimately decide. This helps to reduce unnecessary expectation to deliver because each team is different. 

Why are development teams motivated to deliver?

While the cat’s away the mice will play. (proverb) 

If a manager is not supervising a team, it could be argued that the team will ultimately become demotivated and lose focus and not deliver. 

Within an Agile culture, we look for leaders rather than have people supervisor what other’s do. The product manager job is to motivate the team through the product vision. A team that believes in the product vision is more likely to consistently deliver than a team that is faced with a long list of to-do without any context. The product vision creates a shared value that people aspire to.

What is the benefit of sustained development?

To illustrate the benefits of sustained development we can look at three audiences within the organisation.:


At a company level one of the key benefits to sustained development is a more stable workforce. Having a culture where crunch time is expected may in the short to middle term increase the output by staff as they are working longer hours. In the long run, such a culture leads to burnout with higher staff turnover and staff sickness. In such scenarios, it becomes rather then fully focus on growth the organisation there is a continuous demand to backfill positions. 

One of the best ways to recruit new staff is how current and staff who have left the organisation feel about the organisation. A simple search on Glassdoor on a company’s name can easily attract or repel potential employees. 

A company ability to be agile and pivot quickly in an ever-changing environment is vital. Sustained development is a part of Agile methodology which lends itself perfectly to support MVPs, lean and methodology like Evidence-Based Management. 


A team that is based upon sustained development understands that there is an opportunity cost for any work that is carried out. There needs to be a ruthless focus on the most important because there is limit of what can enter a Sprint.  

Under high-pressure environments, the focus for the team is the here and now. With sustained development time can be allocated to work on future projects because not everything is about completing the backlog. Having time to focus on future projects gives autonomy to the individual to directly influence the direction of the business, and gives variety. 

As a  final example is the idea of removing impediments. An impediment is something that will slow down or stop a developer delivering a story. It could be technical or non-technical, it could be an internal or external factor to the team. Impediments can causes stress to the team members and one of the objectives of the daily stand ups is to highlight impediments. The Scrum master will identify, track and help remove impediments. It is common that the impediment may be removed by the team, but sometimes it is beyond the team. The Scrum master will then work with other teams or external parties to remove the impediments. 


In this section, we will look at two examples. Sustained development on the wellbeing of the individual and how it can help to create a growth mindset. 

  • Managing wellbeing – a team or an organisation that adopts sustained development as part of an Agile framework looks to give autonomy to the employee, a feedback loop as well as a more balanced workload. These features help to reduce the possibility of burnout. 
  • Growing as an individual – in a safe environment where the workload is manageable and everything is not required yesterday gives space to the individual to experiment, learn and take on more responsibilities.

Practical ways to avoid burnout using sustained development 

This section will look at practical ways to avoid burnout. 

  • Avoid making commitments that do not come from the development team. This avoids putting unnecessary strain on the team. 
  • When Sprint planning allocates 10-20% for non-committed time to unknown impromptu meetings etc.
  • Have meetings in blocks rather than spread out through the day. Developers as well as those who create content require uninterrupted focused time. Having meetings spread out may not give the necessary to get into the flow.
  • Make sure there are clear targets for each sprint. 
  • End a sprint on a Friday morning. Have the sprint review and retrospective in the morning and have sprint planning on the following Monday. This leaves Friday afternoon for the team to work on something else. 
  • On the Sprint planning meeting ensure the developers are given time to walk through each story so reasonable know what can be completed. The team should be comfortable saying no.
  • Schedule and ensure time is set aside for the team to improve their technical skills. 
  • After every three Sprints have a week where developers can work on side projects that provide business value. 
  • Ensure the feedback from the retrospective is acted upon to improve the lives of the team. 
  • If not every Sprint, every other Sprint or after a few Sprints the Sprint backlog should contain stories for technical debt and quality of life (QOL)
  • The focus must be placed on impediments that stop or slow down delivery. 
  • Keep the Sprints fun. It helps to bring people together and helps destress potential stressful situations. 

Health Check Model

One way to check the well-being of the development team is to do what is called a health check of the team. An example of a health check can be found on the Spotify engineering blog.

Image used from Pixabay

“In God we trust. All other must bring data” W Edwards Deming


In our previous posts we have consisted Agile as a way to find solutions to complex business problems or Agile as a company culture. The purpose of this post is to look at Evidence-Based Management, an Agile framework that looks to unify an organisation under a single strategic goal and applies Agile processes to work towards achieving that one strategic goal. It was developed and sustained by Ken Schwaber and

Evidence-Based Management and Empiricism

Evidence-Based Management is based upon empiricism and looks to optimise an organisation using an incremental and iterative methodology. Empiricism is a theory that states that all knowledge comes from sensory experience rather than innate ideas or traditions. It was developed in the 17th and 18th Century and expounded by John Locke, George Berkeley and David Hume. Within Evidence-Based Management (EBM) there is no such thing as a guaranteed success , rather everything needs to be tested to prove its value.

What is meant by iterative is doing something again and again with the intention to improve it. Whilst incremental is making minor improvements to an existing product or service. Incremental and iterative go hand in hand as one looks to make minor improvements and the other looks to build on each improvement with another round of Improvements. Empiricism ensures that the improvements are tested to ensure that they actual improve the product or service.

Goals within Evidence-Based Management

EDM similar to North Star Metrics focuses an organisation and its staff under one overarching strategic goal.   Under that single strategic goal there are multiple intermediate and immediate goals to help to pave the way towards fulfilling the strategic goal.

The three different types of goals within EDM are:

  • Strategic goal: this is something that the organisation seeks to achieve that is both aspirational and far in the future. As it is far in the future there are uncertainties, an organisation will create intermediate goals to help navigate the uncertainties.
  • Intermediate goals: these are goals when completed help that the organisation move towards their strategic goal.
  • Immediate Tactical goals: are near term goals for a team or a group of teams which works towards an immediate goal.

 Before embarking on the journey, it is important for an organisation to know its Starting State i.e. where the organisation is relative to the strategic goal when it starts its journey. During the journey it should be aware of its Current State where the organisation is relative to the strategic goal at the present time. It is important to remember that for a goal to be achievable it must be both measurable.

 Progression towards to the Strategic goal

As the strategic goal is far away and the path to achieve it is uncertain, an organisation will run experiments and use the results to determine their next steps. Setbacks are part of the journey and so it may change direction (intermediate and immediate goals) as long as we keep the strategic goal as our destination point.

It is also important to state without have a strategic goal guiding the direction of an organisation it is easy for the organisation to get lost and focus on non-essential areas or vanity goals.

Real-life examples

Using two examples, one in the private sector and another in the charity sector we can illustrate how a strategic goal can guide an organisation’s strategic direction.

Charity sector example

A charity defines its strategic goal to eliminate death by preventable diseases in Kenya, East Africa. That is both a difficult goal to achieve and it is inspirational.

Its intermediate goals could be to:

  • Provide safe drinking water in the cities and villages.
  • Run educational programs to promote good health.
  • Set up and run medical clinics throughout Kenya.

Its tactical goals would take the intermediate goals and break them down into tasks that can be done by the team(s).

As an example, if our intermediate goal is to promote good health through education one of our tactical goals for a team could be:

  • Book a venue for an event.
  • Develop marketing material to promote the event.
  • Find someone to present the event.

Once we run an event, we take the learnings from running the event and look to ways to optimise the future events. Our yardstick for optimisation could be, does the change help to eliminate preventable diseases through education?

By having different teams working on and fulfilling different tactical goals the organisation moves towards fulfilling its intermediate goal of promoting good health. Promoting good health moves the organisation towards its strategic goal of eliminate death from preventable diseases.

Private sector example

Using the private sector let us take the example of a mobile application that helps students organise their college’s lives. It generates revenue and has a product roadmap with several exciting features planned. The organisation decides it is will adopt an EDM framework and defines its strategic goal as:

To ensure all students regardless of background or disability to get the support they need to complete their education.

Let us consider what is the impact on the product road map by adopting this specific strategic goal.

A new proposed feature within the roadmap matches with the intermediate goal and the long strategic goal of the organisation.

  • There is an existing tactical goal can be fulfilled or partial fulfilled by building this feature. The product manager must decide the priority of this feature relative to other items in the backlog.
  • There is no existing tactical goal to match the new proposed feature. The product manager must decide if they should recommend that a new tactical goal be created now or is better to complete the current tactical goals before looking at the possibility of creating new ones.

If the new proposed feature does match any intermediate goals of the organisation it should remain in the product backlog until it can.

The other aspect to consider as it builds it looks to get feedback often and early to ensure what is developed is inline with the user needs.

Measurement within EMB

In the previous section we showed how Evidence-Based Management can be used to determine the goals of the organisation and how through an agile framework of testing we can work towards to fulfilling those goals.

In this section we are going to look at how EMB recommends measuring the value of the organisation and how the organisation can therefore optimise for value.

Organisations may measure value by looking at:

  • Activities: these are things that individuals do. For example, an individual may design an image, write code, attend meetings or perform tasks.
  • Outputs: these are things that the organisation produces. It could be reports, products, new product features or even scientific discoveries.
  • Outcomes: Outcomes are of two types: positive and negative. Positive outcomes happen when our customers or users benefit from using our product or service. For example, establishing a platform for C2C (consumer to consumer) trade like eBay or Etsy allows consumers to trade. A user of these platform benefit being able to generate an income stream. By focusing on the user needs, the positive outcome for the user is also a positive outcome for the company. Negative outcomes are undesirable and can happen when a user is no longer able to achieve something they were previous able to. Sometimes an organisation may retire a product or service which can cause a negative outcome.

EBM is focused on outcomes rather than outputs or activities. It believes that true value is realized by serving the user and not by producing more product or doing more activities. By focusing on the user, we build a product that our users want more then they just need. That in turn improves customer retention and customer referrals.

How is value measured within Evidence-Based Management?


Value can be measured in multiple different ways, some looks at EBITA, others look at ROI whilst others may care about NPS. EBM defines value in four different areas, these are called Key-Value Areas (KVAs). They are:

  • Current Value (CV)
  • Unrealized Value (UV)
  • Ability to Innovate (A2I)
  • Time to Market (T2M)

 Current Value (CV)

The current value is a measure of what the organisation can deliver to its customers and stakeholders today.  Current value is not limited to revenue generated or size of the paid database but also considers the levels of happiness of the customers, the staff and the investors/stakeholders.

Questions they can be used to assess it values could include:

  • How happy are our employees? Would they recommend working there?
  • How happy are our users/customers with our product or service? Is their happiness on the incline or decline?
  • How happy are our investors and stakeholders? Is their happiness on the incline or decline?

By measuring employee happiness, we can optimise for it because an incline or decline in the level of happiness has ramifications all across the organisation. An engaged and happy employee is going to contribute more to the company success then an employee who is unhappy and disengaged with the company.

Levels of user happiness for the product or service acts as an indicator of what needs to be improved in the product to retain customers. Looking at how often a feature is used, and for how long, is a way to understand what a customer value in the product.  It is also important to compare user level of happiness to the past to determine if it is on the incline or decline.

ROI is important for investors but so is the direction and the vision of the organisation. If they are concerns, then they must be addressed. Continuously engaging stakeholders to determine what concerns them and then working to deal with the concerns is essential for the success of an organisation. By measuring for it, we can optimise for it.

It is important that an organisation continuously reassess its position.

Unrealized Value (UV)

Current value measures the value of the organisation today, unrealized value is the potential future value of the organisation. These values represent the difference between what is possible today and what is possible in the future. This difference is the unrealized value.

A start-up may have high unrealized value and a low realized value as it is recently started its journey to greatness. Investing at this stage can help unlock the Unrealized Value and increase the Current Value of the organisation.

An organisation within a mature market that is either a monopoly or an oligopoly may have a high realized value but a low unrealized value as the opportunity to grow is limited.

The type of questions that an organisation can use to assess and reassess its unrealized value could include:

  • Is it possible for the organisation to generate additional value in this market or another market?
  • What is the opportunity cost of pursuing these opportunities?
  • What level of investment would be required to realize the unrealized value?

Time-to-Market (T2M)

An organisation may build an amazing product or service but not until it is available on the market can it deliver value to the customer. Time to market is a measure of how quick an organisation can deliver new products, new services or develop new capabilities. What is important for an organisation is to improve or sustain its time to market.

By maintaining a schedule of regular releases with maybe a smaller number of features per release can improve T2M as well as the Current Value of the organisation relative to having larger releases with more features but release less often.

Having the ability to deliver a product on time but then spending several months fixing the problems that the release created limits the ability for the organisation to launch new products.

Questions that the organisation can ask to assess their T2M can include:

  • How fast can an organisation test a hypothesis with users?
  • How fast can the learnings from the test be understood and adopted by the organisation?
  • How fast can the learnings from the test be implemented?

It is important to optimise the time to market to ensure what is learnt can be implemented in the time frame where it can add value. Being late to market may mean a new feature is no longer consider as adding value but rather becomes what is expected but still important, to no longer required as users has moved to another solution that is more preferable.

 Ability to Innovate (A2I)

Innovation is the ability to improve a product or service for its users. The ability to innovate is a measure of ease at which an organisation can innovate.

The type of questions that an organisation can use to assess and reassess its unrealized value include:

  • What are obstacles that stop an organisation from innovating?
  • What can be done to remove these obstacles?
  • What are the factors which can impacts users from benefiting these innovations?
  • What can be done to ensure users benefit from these innovations?

It is important for organisation that follows a hypothesis, experiment & measure, inspect and adapt model to improve the organisation value. Competitors may be a good example of what is possible, but the real competition is about constantly growing by asking what could you do better.

The interaction between the four values

Realized value is the unrealized value in the past that was achieved and acts as an indicator of how the organisation can innovate and the blockages and stumbling blocks it circumnavigated during its journey.

Unrealized value carries risk as there is no guarantee of unlocking it. By breaking down the unrealized value into tactical and immediate goals as we achieve each goal, we can unlock unrealized value. By adopting a short iterative approach, we can ensure if something does not yield the results we are looking for or is no longer valid, we simply try something else or pivot in a totally different direction. This helps us to achieve our goals and saves time and resources by not pursuing activities that do not yield the optimal results.

Our ability to innovate is how we can unlock unrealized value.  Innovation is seen by some by a single eureka moment. Innovation under an Agile is a process by where we test multiple different hypothesis and look to what works best, and once that is found something we continue to optimise it further through further iterations. The speed of innovation is depending upon our time to market. Optimising time to market, increase the number hypothesis we can test, which helps realize the unrealized value that is available.

To learn more EBM I would recommend reading the guide, which can be found here.

Image by Pexels from Pixabay

 May 7th 2020 marked the first Inside Xbox event to show off gameplay running on the Xbox One X Series, Microsoft’s latest gaming console. There was much hype surrounding this event, unfortunately, many felt disappointed by what was shown. Aaron Greenberg the General Manager of Games Marketing had to say about this:

Is failure bad?

We live in a world where success is celebrated and failure is seen by some as a disaster. Not scoring that point, not passing that exam, or getting that deal can hurt but it can also stop someone from trying again.

 “I have not failed. I’ve just found 1,000 ways that won’t work.” Thomas Edison.

Thomas Edison is regarded as America’s most recognised and prolific inventor. In his pursuit in developing an efficient incandescent lamp, he tried over 1,000 different ways with each one failing, only to finally succeed. Sometimes in life, the greatest opportunity to learn comes from failing, getting up and trying again.

In Agile, failure is just a result of a test, there is no social stigma or negativity around it. By failing and you have learnt something you may not have learnt otherwise.

“What good is an idea if it remains an idea? Try. Experiment. Iterate. Try again. Change the world.” Simon Sinek Together is Better

Failure in an Agile Environment

The purpose of this post is to show how we can leverage user research coupled with a testing mindset to help to exceed user expectation and create a buzz around the Xbox One Series X and its games.

Agile can be defined in two different ways:

  1. The ability to react to an ever-changing environment. When technology introduces new opportunities, markets collapse, or new markets open up, having the ability to react quickly amd take advantage of the opportunity is being agile.
  2. To take what you don’t know (the known unknown) and use a systematic approach to test different hypothesis to make it become  a known known.

In this post we will look at the second approach, that is to develop a set of hypotheses, build a minimum viable product (MVP) to test and use the results for either iterative or deliver to the end users. 

The importance of user research

user research

It is sometimes very easy to forget the maxim,

“you are not your customer”

To help eliminate such biases, user research helps to understand user behaviour, their needs, their motivations through testing, observation and interviewing.

What is user research within Agile?

If Agile is the process by which we can develop and test new ideas, user research is the way we can help generate ideas to test from the users.

What are we looking to test?

Jerret West, CVP of Gaming Marketing at Xbox said, “starting with the May 7th episode of Inside Xbox, we will be showcasing what happens next in the world of Xbox, every month, which is why we’re calling it ‘Xbox 20/20.’

By having an event every month until release provides the perfect opportunity to test and iterate.

To illustrate the idea of using user feedback and iterating we will look at two different examples:

  1. Ray Tracing – a hardware feature for the Xbox Series X.
  2. Halo Infinite – the launch of a new game.


Hardware – Ray Tracing on Xbox Series X

The Xbox Series X is the first Microsoft console to feature Ray Tracing which has traditionally been reserved for high-end PCs.

What questions do users have and where does the information exist?

Gathering information online

In this example, we are looking for users who are discussing ray tracing. This may include content on Quora.comTwitterMicrosoft Answers and the comment section in YouTube videos. What is important is to identify what questions are people asking? What makes people excited about Ray Tracing? And how are people justifying the purchase of a new graphic card or a next-generation console to experience Ray Tracing?

Asking questions – slicing and dicing data 

We have all seen Twitter polls, voting in Instagram Stories, emails with questionnaire etc. there are many ways to get answers to questions. We may want to ask questions to specific audiences which can be done by using different social accounts, time of day of posting or paid postings where we have even more control on who participates. 

What if we took this a stage forward? Microsoft has significant amount of user data on which games are played, which apps are used and which consoles have been purchased by user. We could create different audience segments:

  • Users who have purchased Xbox S and Xbox X.
  • Users who buy multiple games per month.
  • Users who play less than 4 hours of gamer per month.
  • Users by gamerscore. 

It can then be possible to target user in console (having surveys on the dashboard) or over email. Questions can be asked can include if they know what Ray Tracing is? How important are graphics relative to other feathers a new console has to offer? Or if they have any specific questions on Ray Tracing? 

In terms of the audience we can take two approaches:

  1. Select the audience which is most likely to purchase the Xbox Series X and target those.
  2. Look for the audience that share similar properties which is largest in in size relative to other audiences. By optimising a campaign for this audience  will also have the greatest impact on the whole audience.

Developing a hypothesis and MVP

Now that you have data, it is about developing hypothesis from the data and finding, ‘quick and dirty’ ways to test. What is important to remember:

  1. A hypothesis is just an idea, not every idea will work. Don’t become attached to an idea until you have data to prove its validity. 
  2. Your hypothesis may not be the best. 
  3. It is all about getting feedback and iterating.
  4. The solution may not fit all (e.g. ways people learn or consume media.) 
  5. The endpoint is a decision based upon on data you currently have on how to present Ray Tracing for the Series X launch period. Things change over time.  

Let us for a moment walk through an example to see how this could work.  We may for example have a segment of the audience who does not know what Ray Tracing is but regard improved graphics as the most important factor when buying a new generation console. 

We could test showing videos of Ray Tracing on and off versus screenshots of games with and without Ray Tracing, to see which one better communicates how Ray Tracing can make graphics look better. Or is there a place to show both video and images? Taking that a stage further by iterating, what if we had an engineer talk how Ray Tracing works before or after showing the video/images? Rather then using an engineer what if we used a studio head talked about the power of Ray Tracing in building games? The objective here is to test multiple different scenarios with users to find the one that is most optimal. This format can then be used at  the reveal event. 


Launching a New Game – Halo Infinite

Halo infinite

Halo is a system seller and the second biggest IP after Minecraft. We know that Halo Infinite will be shown at the Xbox 20/20 event in July and the game is due to be released this year.

Let’s set ourselves a different question, post the reveal event how quickly can we adapt to user feedback and produce marketing material? Usually, in any marketing campaign, we define all the marketing materials required in advance and have them ready for the event. What if we took a different stance, and decided we needed to be flexible and respond to user feedback during the actual reveal?

The following is a an example how we can build a team that has the capability to respond rapidly to user feedback.


We are going look at two sources to try to predict how fans will react to the reveal. Internal users and historical reactions to new video game reveals.

Internal users

Bring together employees from Microsoft/Xbox who are experienced in storytelling but have not worked on the Halo reveal in a ‘room’ together to show the Halo reveal to. The objective is to highlight potential problems with the reveal.

I would recommend using the format called ‘Braintrust’ pioneered by Pixar, where everyone’s’ opinion is equal, no one is to be blamed for failure and people are encouraged to use Radical Candor to identify the root causes of problems. To learn more about Braintrust I would suggest reading this article on or reading the book Culture Code by Daniel Coyle

Having this critical feedback can be used to identify which problems need to be resolved before the reveal and which are a potential problem that we need to help mitigate during the reveal event. 

Historical reactions

There have been many First Person Shooters in the past, there include previous releases of Halo including the more recent release of Halo Master Chief Collection on the PC. There have had reveals, trailers, gameplay videos with lots of user feedback. Take all that user feedback and identify common questions, common disappointments, recommendation I.e. ‘I wish we could see more of….’

Building a multi-disciplined team

The idea is to put together a team with different skills so they can act independently from all the  other teams. The team should consist of people who have the necessary skill and experience to produce any marketing material  they require. The team may include graphic designers, video designers, content writers, engineers, video editors etc. The team should be independent and able to self organise in a way that works for them. In software development we would  call this team a Scrum team. In our example we can say we have an Agile team. 

Their first objective is to understand what the Halo Infinite reveal contains, take the feedback from the internal users and the data from the historical reactions and understand possible pitfalls.

Building a content library

Once you have all these possible scenarios, the scenarios need to be ranked according to the chance of it happening on the day of the reveal and the effort required to produce something. Let’s look at a possible example of this playing out.

Let’s say the July event is 90 minutes long. Halo Infinite is shown in the first 5 minutes and there is space reserved for the last 5 minutes for a follow up to the Halo Infinite reveal. Within minutes of the reveal the feedback online is negative indicating an area of disappointment, for example, the lack of multiplayer gameplay. The team has 80 minutes event to rectify it and present something at the end of the show.

It may seem scary at first producing a something in a very limited time-space but with all things, it is all about practice and having retrospectives to see how you can improve. Questions that could come up in the retrospective could include:

  • What additional content needs to be added to the content library, that we can repurpose? 
  • Which Halo staff need to available to interview on the day if needed?
  • What skills does the team need to learn?
  • How does the team perform under pressure?
  • What can we do to ease the pressue?
  • What are the impediments that need to be removed?
  • What backs up can we put in place?

Each time you run a scenario the team become quicker because they know what works.  Through the retrospective they have highlighted the bottlenecks and subsquent dealt with them. And on the day of the reveal and they become aware of what is needed, they function like a well oiled machine, relaying on each other’s capability to deal with what ever hit’s them. 


The post looked at two different ways we can apply a framework of taking user feedback, testing and optimising as we learn. What is key to note, there is no absolute right solution, it taking the data you have, making educated guesses and testing with real users. When you build a business that Agile in both senses it will learn from it mistakes and takes it successes and look to amplify them.

Image Credit

Thanks to Zohre Nemati for sharing their work on Unsplash.

Photo by David Travis on Unsplash


The world of today, is not the world of yesterday and the world tomorrow is not the world of today. The rapid growth in technology, communication and changes in society has meant that traditional businesses which  stood the sands of times have crumbled away and new businesses have taken their place. And even these businesses with eventually crumble as innovation and disruption creates new opportunities and new businesses.

To be successful in the 21st Century a business needs to be nimble enough to innovate and close enough to its customers to listen. The objective of this post is look at one possible methodology the Agile framework and how it can foster an innovative culture.

A definition of culture :

 “the total sum of the values, customs, traditions, and meanings that make a company unique.”

What is Waterfall methodology?

To understand Agile, it is important to understand the Waterfall methodology. Waterfall methodology is also referred to as the linear-sequential life cycle model which focuses on completing a task before moving on to the next phase. Waterfall methodology works well when requirements are well documented, the product definition and technology is stable, and the project is short. Where Waterfall struggles is when requirements are not fully known,  product definition or technology regularly changes. This can lead to significantly delays and costs that escalate.

Early in the 2000s, Agile was conceived for software development as an alternative to software development lifecycles methodologies such as Waterfall.

What is Agile?

Agile is an iterative approach to software development using empirical evidence. There are three pillars to empirical evidence:

Transparency: This means presenting the facts as is. All people involved—the customer, the CEO, individual contributors—are transparent in their day-to-day dealings with others. They all trust each other, and they have the courage to keep each other abreast of good news as well as bad news. Everyone strives and collectively collaborates for the common organizational objective, and no one has any hidden agenda.

Inspection: Inspection in this context is not an inspection by an inspector or an auditor but an inspection by everyone…. The inspection can be done for the product, processes, people aspects, practices, and continuous improvements….

Adaptation: Adaptation in this context is about continuous improvement, the ability to adapt based on the results of the inspection. Everyone in the organization must ask this question regularly: Are we better off than yesterday? source 

The focus is to deliver functional code at the end of each Sprint. A Sprint typically lasts two weeks. The Sprint begins by agreeing on what work can be accomplished within the Sprint from a backlog of tasks. The objective of the Sprint is to deliver functional code this can be rolled out to users to be tested. By getting the code in front of users, feedback can help shape future Sprints.

Agile software development principles

The Manifesto for Agile Software Development is based on twelve principles

  1. Customer satisfaction by early and continuous delivery of valuable software.
  2. Welcome changing requirements, even in late development.
  3. Deliver working software frequently (weeks rather than months).
  4. Close, daily cooperation between businesspeople and developers.
  5. Projects are built around motivated individuals, who should be trusted.
  6. Face-to-face conversation is the best form of communication (co-location).
  7. Working software is the primary measure of progress.
  8. Sustainable development, able to maintain a constant pace.
  9. Continuous attention to technical excellence and good design.
  10. Simplicity—the art of maximizing the amount of work not done—is essential.
  11. Best architectures, requirements, and designs emerge from self-organizing teams.
  12. Regularly, the team reflects on how to become more effective and adjusts accordingly.

The benefits of Agile within software development 

  1. Speed to market – By delivering working code at the end of each Sprint and embracing ideas like MVP allows the product to get into the hands of its potential users in weeks, rather than months.
  2. Higher customer satisfaction – engaging users and taking feedback that is incorporated into the product backlog can produce software or features that users actual want.
  3. Control – Agile teams are self-organising and responsible. They establish their own ways of working together based upon their context, experience and any other organisational constraints. They don’t require a manager and have the trust that they will get the job done. This has been shown to motivate teams to push the boundaries.
  4. Higher quality output– getting daily feedback and testing regularly reduces mistakes and oversights.
  5. Transparency – providing visibility in the product backlog and active Sprint creates accountability, helps eliminate office politics and provides visibility to the organisation.
  6. Velocity – The daily stand-ups, the retrospectives help eliminate impediments which in turn increases the velocity of the team.


Examples of Agile methodology outside software development 

We have covered the principles and the benefits of Agile in software development, we can now want to look at how an Agile framework can be utilised in marketing and product development.

Growth Hacking 

Sean Ellis coined the term ‘growth hacking’ in a blog post in 2010. A growth hacker is focused one single metric, how to grow a business. Using an Agile framework, in each Sprint the cross functional team develop, prioritise and test ideas for growth. The results from these tests are then used to further develop ideas to test i.e. an iterative approach. Those interested to learn more should read the growth hacking blog at

Design Sprints

Developed by Google Venture the design sprint is a “five-day process for answering critical business questions through design, prototyping, and testing ideas with customers.” source

A Design Sprint is split across five days with time allocated to:

  1. Defining the problem you wish to solve.
  2. Designing possible solutions.
  3. Creating a hypothesis for testing.
  4. Developing a high-fidelity prototype.
  5. Testing with users.

What makes Designs Sprint successful can be summarised in three factors:

  1. Using time-boxed events to streamline defining the problem and developing possible solutions.
  2. Creating an MVP (Minimum Viable Product) from one of the possible solutions.
  3. Testing the MVP with real user to measure success.

To learn more about Design Sprint I would recommend reading Jake Knapp book Sprint: How to solve big problems and test new ideas in just five days. Or read the post on Lego’s implementation Design Sprints at scale

Minimum Viable Product (MVP) vs Minimal Desirable Product (MDP)

A minimal viable product (MVP) is a technique where a new product is created with the least effort possible used to validate a hypothesis. MVPs allows for rapid learning at the lowest possible cost. The concept of MVP was popularized by Eric Ries, the author of the Lean Startup.  He explains an MVP as,

“the idea of the minimum viable product is useful because you can say: our vision is to build a product that solves this core problem for customers and we think that for the people who are early adopters for this kind of solution, they will be the most forgiving. And they will fill in their minds the features that aren’t quite there if we give them the core, tent-pole features that point the direction of where we’re trying to go.

So, the minimum viable product is that product which has just those features (and no more) that allows you to ship a product that resonates with early adopters; some of whom will pay you money or give you feedback.”

A minimal desirable product (MDP) is focused on testing a great user/product experience with the least effort required to test. For example, creating a high-fidelity prototype would be an example of a minimal desirable product used for testing.

Andrew Chen explains the MDP process as

  • Understand user goals
  • Create a Minimum Desirable Product
  • Listen to users and maximize Love
  • Iterate to a great product experience

North Star Metric

The North Star Metric is a concept that has emerged from companies from Silicon Valley who invested in long term sustainable growth by creating and optimising that ‘a-ha’ moment with their customers. The North Star Metric is a single metric that focuses on the product’s core value. It defines the relationship between the customer problems that the product team is trying to solve (the ‘a-ha’ moment) and then the revenue that the business aims to generate by doing so. It has helped teams move beyond focusing on surface-level growth to long term customer growth as everyone and everything is focused on a single metric.

A North Star Metric may not be the flashiest number, nor is it a vanity metric, such as Facebook likes or Twitter followers. Getting one hundred new Twitter followers doesn’t equal growth. Likewise, focusing all your effort on free trial signup, will not provide insight into whether those people will actually use your product, or whether they’ll stick around when the free trial period has ended. North Star Metric is a leading (not lagging) indicator of a future business outcome that your company cares about.

Why do you need a North Star Metric

The North Star Metric provides three essential benefits:

  1. It provides the company and its staff clarity and alignment on what needs to be optimised and what can be left alone.
  2. It communicates in a simple single metric the progress of the product to the whole company.
  3. It holds the company accountable for an outcome.

An Agile Culture

In this section, I want to look at how concepts of Agile can be used to define the culture of a company. By Agile I mean:

  1. Using data over opinion – a culture that looks at market trends, carries our user research or analyses user data to make decisions over personal opinions can be seen as a culture that has adopted concepts of Agile.
  2. Willing to test to find answers – rather than make decisions based upon on experience or decision by committee, an Agile culture focuses on validating a hypothesis.
  3. Willing to fail – a company who is willing to test must be willing to learn when things go wrong. There is much to be learnt from failure if your is willing to listen.
  4. Adapt to change – what worked yesterday, may not work today and tomorrow could be considered to be detrimental to the organisation. A willingness to change is essential for an Agile culture.
  5. Start small and grow – Before allocating resources to anything, it is about iteratively testing to look for those 5mm gains that can scale.
  6. Self-organising – an Agile culture gives autonomy to staff to make decisions and take risks.
  7. Collaboration – an Agile culture encourages staff to share their learnings and collaborate to help move the business forward.
  8. Flat structure – rather than having multiple levels of hierarchy and slowing down the decision-making process, Agile promotes a flat structure empowering a team to make decisions.
  9. Performance Orientated – An Agile culture see success not as single decision or action but rather building a team who continuously strive for success. It is this relentless pursuit that drives performance.
  10. Transparency – an Agile culture fosters a culture of sharing information and best practises, results are documented and shared throughout the organisation so the learnings can benefit all.
  11. Servant leadership – happens when a leader primary function is to serve, putting the needs of the employees first. When staff feel valued and trusted they are motivated to perform at their best.
  12. Standard ways of working – facilitate integration, including common languages, processes and meeting formats help to improve communication between teams

An example of an Agile Culture – Spotify’s’ Squad, Tribes and Guilds  

Spotify the popular music app was launched in 2008, they owe part of their success to Agile methodology. As they scaled the business, they scaled the way they used Agile. They believe to have rules at the start and then later break them to adapt to the team’s need, giving them a firm foundation with the ability to adapt based upon needs.

The method they developed is known as Spotify Tribes. Spotify organisation is structured into Squads, Tribes, Chapter and Guilds.

Squads – Spotify staff are divided into teams consisting of 6-12 people focusing on one feature area. Each squad acts like a start-up, they are autonomous and accountable. They use MVP to release often. They resemble the Scrum team as they have Product Owner and an Agile coach,  front and back end developer to deliver.

Tribes – Squads working on a related feature area form a tribe. A tribe may consist of 40 – 50 members with an upper limit of 100.

Chapter – A chapter consists of individuals from various tribes at the horizontal level of the functional organisation.

Guild – an informal group of people from different tribes who have a common interest.

The chapters and guilds allow for cross-training, problem-solving and aligning between teams.

Within the Spotify model Agile, each squad has the autonomy to decide what they would like to build, how to build it and how they will work together. Squads will also communicate with other squads to help develops solutions. As the squad is part of a tribe, there needs to be alignment with the mission, product strategy and the short-term goals.

Developing an Agile Culture 

Organisations are different because they exist to fulfil different needs, exist in different verticals and have different histories. Some may be public, some may be private, some may hire less than 5 people, whilst others will have over 100,000 staff.

Developing an Agile culture can be initiated by internal or external factors.

What factors may cause a culture to change?

External factors can include:

  1. Economic
  2. Legal
  3. Changes in society.
  4. Technological changes especially when it is disruptive.
  5. The availability of information.

Internal factors can include:

  1. Change in management.
  2. Internal dissatisfaction including high staff overturn.
  3. Increasing losses or dwindling profits.
  4. Hiring decisions.

Ways to bring about a cultural change 

In this section we want to look at two different ways we can bring cultural change:

  1. Use a top-down approach when there is change is instigated from the highest level of management within the organisation.
  2. The bottom-up approach where you start from the grass roots to prove the value of the Agile framework and grow.

Taking a top-down approach

A top-down approach occurs when changes are made from the highest level of management which then impacts every individual throughout the organisation. Changes to culture and the direction of the organisation are complicated and can face resistance at many levels, so a key feature is a buy-in at all levels.

One of the modern-day approaches to get buy-in is to galvanise the direction of the organisation by defining the why of the organisation. The why or the purpose of the organisation helps to create a sense of belonging and meaning which is key to culture change.

The video below is Simon Sinek talking about starting with why.

An inclusive approach to determining the Why of the organisation is to allow staff from across the organisation to participate in workshops to share their uncerstanding. Everyone has a perspective and by sharing each other’s understand can lead to a common understanding of what the organisation stands for. To learn more I would suggest reading Find You Why by Simon Sinek. Once the organisation has a reason for being (it’s why) it needs a way to measure if it is moving in that direction. That’s where the idea of North Star Metric comes in. The North Start is that one metric that matter most to fulfil the why of the business.

At an organisation level, it is about setting OKRs which then feed into the North Star Metrics. Staff and then organised cross-functional teams and give them the freedom to define how to achieve their OKRs. At this level it is about fostering a mindset that will think:

  • do not know the answer, but I will iterate until I find it.
  • I will provide visibility so as I learn the organisation will learn.
  • I will be authentic in my communication and being.
  • I will critically learn from the mistakes I make.
  • I am responsible for all our successes.
  • I will focus on the things that move us closer to the North Star Metric and not move away.

The fostering can be achieved through:

  • Having Agile coaches who support the transformation.
  • Define expectations.
  • Removing impediments or obstacles which delay or stop progression.
  • Lead by example. Organisational change comes from when what is spoken matches what is believed in the heart and acted upon.
  • Reward staff for innovating and working smart and not working long hours and achieve mediocre results.
  • Over-communicate the organisation purpose, expectations and alignment of people though Agile practices.
  • Establish trust throughout the organization.
  • Hire the people who have the right culture fit i.e. they believe in the same values of the organization.
  • Celebrate success.

Taking a bottom-up approach

This approach is about starting small, proving the value of the Agile framework to the organisation and scaling.

It can be started by identify an area of the business that you want to improve. Define the one key metric that matters the most to that business area. It could be a performance metric, a revenue number or something else. As long it can be measured and it is important to that area of the business.

To create a single cross-functional team who have the expertise to make it happen, doesn’t even need to be their full-time job. What is required is that the members of the team will need to:

  • Understand what and how an Agile framework works.
  • Be able to implement Agile framework within an organisation that follows a different approach.
  • Accept that data is more important than opinion.
  • Able to motivate and lead and effectively work within a cross-functional team where there is no hierarchy structure.
  • Be able to learn from failure.

It may take several sprints to achieve success and tools such as GV Design Sprint can be utilised. What is important from every success or failure, something is learnt. Some of the best learning can be derived from what failed. Every test, every result, all learning should be documented and shared within the organisation.

Once you have a string of success. It is needs to be decided how do you best communicate your achievements? Who do you show, and how do you show your results? Some may want a summary, some may want details, some think in numbers and graph, others on the impact on the bottom line.

To scale Agile within the organisation it is about understanding what resources are required to scale and then working with management, business leaders to acquire those resources.

In both the top-down or bottom-up approach the following three factors needs to be taken into consideration:

  1.  Focus on the customer and their needs and wants.
  2.  Keep communication open between teams. Communicate regularly and early on. With larger teams this is especially important as there are dependencies and opportunity if the teams collaborate and work together.
  3. Keep an eye open for new opportunities or new threats and be agile and adapt.


Agile as a framework is based upon principles but at its core it is about trusting staff to self organise and make decisions on what should be built and tested. An Agile culture takes those principles and applies it to the organisation as a whole, giving the staff the freedom to innovate, to fail and to learn. Through testing, success can be uncovered and amplified iterative testing other possible solutions.

Photo Credits

Micheile Henderson
Pietro Mattia
Alvaro Reyes
Mike Setchell
Ross Findon

The purpose of this post is to provide ideas on how Microsoft can leverage the powering of streaming (xCloud) to gain access to the two billion gamer who game on mobile, tablet devices or other devices.

Mobile Gaming and its Significance

According to a report published by mobile gaming represented 18% of total gaming revenue in 2012, by 2014 it had grown grew to 34%. Mobile gaming revenue is estimated to represent $70.3 billion in 2018 nearly double console game revenue in the same year.

According to a report by gamers play on average 5.96 hours per week with mobile devices globally being the predominate device of play.

Gaining Traction in the Mobile Space

As the market is moving toward mobile gaming, companies like Sony, Google and according to rumours Amazon and Verizon are all investing in game streaming technology. Microsoft could take a number of different paths to gain traction within the mobile gaming space that include:

  • Microsoft could produce games on Android and iOS, an example of this would be Gears Pop.
  • Become a platform holder, the Microsoft Phones was an example of this position.
  • Providing services to the mobile space. Services like Azure, Xbox live or the Havoc engine can be used my mobile game developers.
  • Provide a cloud gaming platform that runs on mobile devices.  Microsoft Project xCloud is an example of this.

What is Project xCloud?

At E3 2018, Phil Spencer Microsoft VP of gaming announced a cloud gaming/streaming project that Microsoft has been actively working on.



Project xCloud is enabling Console Native games to stream through our Azure-hosted game servers and streaming clients. Any Console Native game currently shipping in the Microsoft Store on Xbox will be capable of streaming to a mobile device. Project xCloud is an open platform with a customizable Client UX where streaming starts with Xbox game developers not having to modify a single line of existing game code. source

Competitor Space – Google Stadia


Google in March 2019 announced their cloud gaming platform – Google Stadia. 

Engaging Mobile Gamers by leveraging Project xCloud

I am going to use the AIDA model developed by E. St. Elmo Lewis to illustrate how Microsoft could create engagement with gamers. 

Introducing the Purchase Funnel – AIDA

The AIDA model is used to describe the steps or stages a consumer goes through to make a purchase. The stages are:

  • AAwareness – the customer is aware of the existence of a product or service.
  • IInterest – actively expressing an interest in a product group.
  • DDesire – aspiring to a particular brand or product.
  • AAction – taking the next step towards purchasing the chosen product.

Creating Awareness of Project xCloud – Gaming with You at the Centre

Using Google Trends comparing search volume for Google Stadia vs xCloud shows that Google achieved significantly more search volume/interest for the Google Stadia platform then Microsoft has for Project xCloud in the last 12 months.

Building awareness is essential when a product or service is new and unknown in the market. Here are some suggestions to create awareness.

Idea 1 – xCloud Event

  • Separate the presentation on Project xCloud from other Xbox announcements at E3.  Have its own event, use a time to achieve maximum day coverage throughout the world.  For example, 9am PST, is 12pm EST, 5pm GMT, 6pm CET, 9pm GST and 9pm IST. It may not need to be live or in front of an audience,  reach is more important.
  • A separate event allows for breadth and depth within the presentation. Media can focus on reporting on xCloud, rather then juggle xCloud, new hardware, new studios, new IPs or in depth story about Halo/Gears of War.  
  • A separate event also gives marketing an opportunity to build top of mind awareness for the event and project xCloud. The focus of the event should be:
    • Why game in the cloud?
    • What makes Microsoft solution unique?
    • What games do they expect to see?
    • What can they do to learn more about xCloud? Get an email address or a follow in Social Media.
    • The road map for the future i.e. what to expect.
  • Post event, measure online sentiment, search volume and ensure the conversation continues.  Marketing tools such as YouTube Ads, Banner ads, ads over social media, press releases, exclusive interviews or outdoor advertising can be all used to keep the conversation going.

Idea 2 – Website

Searching on Google or Bing for the keyword ‘project x-cloud’ brings up a blog post from the Microsoft blog site and a news posts from the Xbox site in position 1 of the SERPs.

As of April 2019, I could not find a sub-domain or an entire site or a social media handle owned by Microsoft about Project X-Cloud. Having a website and being social activity has numerous benefits which are established and beyond the scope of this article.  

Idea 3 – Mobile Game Discovery

A typical model for mobile gaming:

Discovery -> Download -> Play a freemium game -> Pay

What if we took a existing game or created a game title that was optimised for mobile devices (included touch control), wrapped it in an app and launched on the App Store. Optimise the app to rank within the store and marketing the game (banner advertising, influencers, reviews etc).

The marketing model would be:

Discovery -> Download -> Play for limited time -> Pay subscription for that title to unlock game.

Now that we have a paying user for single title, it would be all about upselling full access to xCloud or selling access to other games. 

Creating Interest for xCloud

Awareness is the doorway to interest. The purpose of this section is to how to generate interest for xCloud.

Idea 1 – Demos and Previews

Let users see the technology running on their devices. This will allow them to actualise what is possible. Give access to press, influencers to generate reach and hype and then the general public.

Idea 2 – Hands on Events

Organise hands on event in major cities. This will allow users to try the technology and help maintain top of the mind awareness.

Idea 3 – Content Marketing

Using different content types (audio/video, written, visual, Q&A) develop material on the following areas:

  1. What is Cloud Gaming?
  2. How does it work?
  3. Will it work on my Internet connection? (Giving the ability to test)
  4. Can I play whilst I am travelling?
  5. What do I need to play? And how do I sign up?
  6. Which devices will it work on?
  7. Is it safe for my children to play and how do I ensure they remain safe online?
  8. Can I play on my TV?
  9. Can I play co-op?
  10. What kind of games can I play?
  11. The cost of freemium games vs xCloud.
  12. How much does it cost?

Creating Desire

The purpose of this section is to provide recommendations to help create desire for xCloud.

Idea 1 – Leveraging Existing Titles 

Take a popular mobile game and develop and optimise for the xCloud. This could be improved AI, better graphics, the removal of freemium content, new and unique content or improved multiplayer/co-op options.  

Idea 2 – Cost Calculator

Playing a AAA games at its highest fidelity requires investment, especially if you looking to play it on a PC. Over time additional investment will be required. Having a simple cost saving calculator showing how much money will be saved over time will help build desire, Also including other benefits to cloud gaming e.g. time to install, hardware scaling to meeting the demand of developers, freedom to choose where and how you play will also help.


Idea 3 – Playing Together


Gaming can be very social. Make it easy for users in a single location to share a device eg a 4K TV, tablet (e.g. split screen gaming),  allow users including guests to drop in with a simple sign in process so they can play together. Save the set-up to make it easy to replicate in the future.  Taking this a step further; based upon the users and the location recommend which games to play, pick up from a previous game played game, keep score on a leader board, create videos of key moments to share (scoring a goal or point in a sports game).

Idea 4 – Game Challenges 

Allow gamers and game manufacturer to set up unique challenges to test the players skills or for fun by sharing a link to a game in play. For example having a baseball game in the last round when you team is down, or a fighting an end of level boss whose difficulties has been tweaked. The idea is to take an existing game and extend the experience for players.  

Idea 5 – Game Demos

For an Xbox player having the ability to demo a game on the cloud before buying the physically or digitally helps achieve two things:

  1. Help the user make an informed decision about the game.
  2. Let a user experience the benefits of cloud gaming.

Idea 6 – USPs, IPs and Gaming Franchises

When consumers have the option to choose from multiple different cloud gaming services, every distinguishing factor can lead to a competitive advantage for one service over another. There is no doubt a soccer game like Fifa will be incredibly popular on the cloud especially if cross-play is supported, but in many ways seen as must have. An exclusive game can be seen to an competitive advantage as it helps create desire.

Some gamers would be interested in Marvel games, some would want remakes or sequels, some new IPs and others new experience that could only be possible due to the power of the cloud. It is all about understanding what consumers are looking for and providing titles to match that desire. 

Like the stock market you make an investment today with the belief that tomorrow would be even more bountiful. A platform like xCloud needs a strong catalogue of games across the various genres, a transparent pipeline for games in the short term and an understanding of the road map for the future.

Idea 7 – New Types of Games

Taking this genre of games further. Subscription services offers a way to fund games that may not exist in the freemium or the outright purchase model. Cross referencing games that are played vs the audiences that play them could reveal genres that need to be strengthened or types of games that should exist.  This insight can then be used to build games that may not exist without a subscription model. Educational games could be one example of this.  

Idea 8 – Social Media                            

With user consent show which xCloud games are popular amongst your friends on social media. It could as simple, your friend X is playing Y would you like to join them? Make it easy for them to try a game. 

Idea 9 – Make it Easy to Set Up

One of the use cases for xCloud is to be able to game on a home television without the need of a console. From a user perspective the application should be readily available on Smart TVs, TV dongles or Android boxes. From a marketing perspective is all about convey how easy it is to get up and running.

Taking Action

The purpose of the final section is to look at ideas on how to move the users from desire to taking action. Taking action is to provide a compelling reason to buy and removing any lingering doubts.

Idea 1 – Tailoring the Offer

Not one shoe size fits all. Some users would require a free trail, other a heavy discounted first month, others a discount over a longer period. Some would only want to pay when they play (pay per day), others monthly, other quarterly/yearly. It is all about testing and giving a user the ability to build a bundle that works for them. At the same time, it is important to consider that too many options can lead to confusion which reduces the chance of a sale.

Idea 2 – Bundling Offers

Bundling or providing offers are a great way to provide value to a potential customer and adds additional leverage to help acquire a paying customer. Examples could include:

  • Buy 12 months of xCloud and get a free digital Xbox game worth $60
  • Buy the Deluxe of a game and get 3 months of xCloud.
  • Gift a month of XCloud to a friend and receive a month for free.
  • Buy 6 month of xCloud get 6 months of Xbox Live for free.
  • Buy a game and get the xCloud edition of the game for free.

Idea 3 – Set up costs 

Owning an Xbox controller could be a barrier to signing up. It could be the price of a controller or the size of the controller relative the size of a mobile phone. Including the cost of the controller within the monthly fee of xCloud or offer a 50% discount on a controller from Xbox Design Lab could help the cost more attractive to a potential xCloud user. Having a smaller Xbox controller may appeal to some users as it will be easier to carry and use whilst travelling.

Idea 4 – Mobile Data

Playing on the go using a mobile network will consume data from the user’s phone package. Build relationships with mobile phone networks to provide uncapped data on mobile phones for gaming on xCloud. This will help remove a barrier to entry. 


I have outlined a number of different strategies that can be employed to help Microsoft grow the user base for xCloud. With every idea it is all about testing and taking the learning to iteratively improve. 

The challenges of running a business

Running a business has always been a challenge and especially today with all the possibilities and challenges that globalisation creates. Businesses can run 24/7, 365 days a year so having real time or near real time data helps in decision making, but the number of different metrics available can make it difficult to decide what to optimise for. Should it be revenue? Repeat business? Time spent of Site? Facebook Likes?

Optimising for some KPIs may have little to no impact on the success of the business. Sometimes different departments within a company may choose different KPIs to optimise for, but these departmental KPIs could negatively impact other departments. Simplifying metrics and creating an overarching metric not only makes decision making easier, it also help focus a company in a single direction.  This is where the idea of a North Star Metric comes in.


Product Core Value and an ‘a-ha’ moment

An ‘a-ha’ moment is a moment of sudden realization, inspiration, insight or comprehension, when something was unclear or uncertain becomes clear, we call that an ‘a-ha’ moment.  For example, when a customer realizes that your product resolves a problem that they are facing they have an ‘a-ha’ moment, they see can see how the core value of a product helps resolve their problem.

To illustrate with an example let’s take a look at Instagram.

Once the founders identified Burbn/Instagram would be all about photographs, they removed all other functions in the app. Photo and photo sharing became the core value of the product. Instagram launched and within a few hours became the number app in the app store.

Imagine for example Instagram after becoming the number one photo app focused on generating revenue rather than optimising and improving its core product, would it still exist today? This is where the idea of a North Star Metric comes in; it focuses a company to optimise their core value over any other priority. Businesses which optimise for their core value have great customer engagement, lower acquisition costs, higher retention rates and stronger referral rates.


What is a North Star Metric?

The North Star also known as Polaris sits almost directly about the North Pole. It isn’t the brightest star but before we had GPS or even road signs, the North Pole would be a reliable indicator of where North is, and once you know where North is you can calculate where South, East and West. Sailors or travellers once they located the North Star in the sky that could reach their destination safety.

The North Star Metric is a concept that has emerged from companies from Silicon Valley who invested in long term sustainable growth by creating and optimising that ‘a-ha’ moment with their customers. The North Star Metric is a single metric that focuses on the product’s core value. It defines the relationship between the customer problems that the product team is trying to solve (the ‘a-ha’ moment) and then the revenue that the business aims to generate by doing so. It has helped teams move beyond focusing on surface-level growth to long term customer growth as everyone and everything is focused on a single metric.

A North Star Metric may not be the flashiest number, nor is it a vanity metric, such as Facebook likes or Twitter followers. Getting one hundred new Twitter followers doesn’t equal growth. Likewise, focusing all your effort on a free trial signup, will not provide insight whether those people will actually use your product, or whether they’ll stick around when the free trial period has ended. North Star Metric is a leading (not lagging) indicator of a future business outcome that your company cares about.


What do you need a North Star Metric?

The North Star Metric provide three essential benefits.

  • It provides the company and its staff clarity and alignment on what needs to be optimised and what can be left alone.
  • It communicates in a simple single metric the progress of the product to the whole company.
  • It holds the company accountable to an outcome.


A north star metric should consist of 2 parts:

  • A statement of your product vision
  • A metric that serves as a key measure of your current product strategy.


Examples of North Star Metrics


Before we provide some guidance on how to define your own North Star Metrics I believe it would be beneficial to provide examples of different North Star Metrics.

Amplitude on their blog in an article called Product North Star Metric define their vision, their Key metric and North Star Metrics.



Core value: Connecting people who need a place with people who can host.

North Star Metric: Nights Booked.



Core value: Build Social Value

North Star Metric: Daily Active Users.



Core value: Facilitate the sharing of knowledge in the world.

North Star Metric = Number of questions answered.



Core value: Online shopping made easy.

North Star Metric = Sales



Core Value: Where people share ideas and stories.

North Star Metric: Total Time Reading.


Other examples of North Star Metrics include a real estate agent: number of open houses, restaurant: average meal check size, or a car salesman with number of test drives per day.


How do you define your own North Star Metric?

“Your product north star should be specific to your product and what your customers value.” Amplitude

To understand what your North Star Metric should be, look at how your product adds value to your customers. The data could be qualitative or quantitively. What is the one thing that they would miss the most if the product no longer existed?

Product Manager for Growth at GrowthHackers Hila Qu gives five points to keep in mind when selecting your North Star Metric:

  1. The metric should be used to measure if a user has experienced the core value of your product.
  2. It should reflect user’s engagement and activity level. The more they are engaged the higher the value, and vice versa.
  3. It is the single metric needed to indicate that the business is heading in the right direction.
  4. The metric should be easy to understand.
  5. It may not be possible to have the perfect North Star Metric. What you are trying is to find here is a metric that makes the most sense for the entire business to focus on. It might take a few iterations to finally find the right one.

Areas of the business that can help define what the North Star Metric could be, can come from the customers, the customer service team, the sales teams, analytics, marketing material or even a competitor.

When looking for a North Star Metric it should be the focal point of the business, a statement of your product vision, a persistent metric for real growth opposite to a vanity metric.