“In God we trust. All other must bring data” W Edwards Deming
In our previous posts we have consisted Agile as a way to find solutions to complex business problems or Agile as a company culture. The purpose of this post is to look at Evidence-Based Management, an Agile framework that looks to unify an organisation under a single strategic goal and applies Agile processes to work towards achieving that one strategic goal. It was developed and sustained by Ken Schwaber and Scrum.org.
Evidence-Based Management and Empiricism
Evidence-Based Management is based upon empiricism and looks to optimise an organisation using an incremental and iterative methodology. Empiricism is a theory that states that all knowledge comes from sensory experience rather than innate ideas or traditions. It was developed in the 17th and 18th Century and expounded by John Locke, George Berkeley and David Hume. Within Evidence-Based Management (EBM) there is no such thing as a guaranteed success , rather everything needs to be tested to prove its value.
What is meant by iterative is doing something again and again with the intention to improve it. Whilst incremental is making minor improvements to an existing product or service. Incremental and iterative go hand in hand as one looks to make minor improvements and the other looks to build on each improvement with another round of Improvements. Empiricism ensures that the improvements are tested to ensure that they actual improve the product or service.
Goals within Evidence-Based Management
EDM similar to North Star Metrics focuses an organisation and its staff under one overarching strategic goal. Under that single strategic goal there are multiple intermediate and immediate goals to help to pave the way towards fulfilling the strategic goal.
The three different types of goals within EDM are:
- Strategic goal: this is something that the organisation seeks to achieve that is both aspirational and far in the future. As it is far in the future there are uncertainties, an organisation will create intermediate goals to help navigate the uncertainties.
- Intermediate goals: these are goals when completed help that the organisation move towards their strategic goal.
- Immediate Tactical goals: are near term goals for a team or a group of teams which works towards an immediate goal.
Before embarking on the journey, it is important for an organisation to know its Starting State i.e. where the organisation is relative to the strategic goal when it starts its journey. During the journey it should be aware of its Current State where the organisation is relative to the strategic goal at the present time. It is important to remember that for a goal to be achievable it must be both measurable.
Progression towards to the Strategic goal
As the strategic goal is far away and the path to achieve it is uncertain, an organisation will run experiments and use the results to determine their next steps. Setbacks are part of the journey and so it may change direction (intermediate and immediate goals) as long as we keep the strategic goal as our destination point.
It is also important to state without have a strategic goal guiding the direction of an organisation it is easy for the organisation to get lost and focus on non-essential areas or vanity goals.
Real-life examples
Using two examples, one in the private sector and another in the charity sector we can illustrate how a strategic goal can guide an organisation’s strategic direction.
Charity sector example
A charity defines its strategic goal to eliminate death by preventable diseases in Kenya, East Africa. That is both a difficult goal to achieve and it is inspirational.
Its intermediate goals could be to:
- Provide safe drinking water in the cities and villages.
- Run educational programs to promote good health.
- Set up and run medical clinics throughout Kenya.
Its tactical goals would take the intermediate goals and break them down into tasks that can be done by the team(s).
As an example, if our intermediate goal is to promote good health through education one of our tactical goals for a team could be:
- Book a venue for an event.
- Develop marketing material to promote the event.
- Find someone to present the event.
Once we run an event, we take the learnings from running the event and look to ways to optimise the future events. Our yardstick for optimisation could be, does the change help to eliminate preventable diseases through education?
By having different teams working on and fulfilling different tactical goals the organisation moves towards fulfilling its intermediate goal of promoting good health. Promoting good health moves the organisation towards its strategic goal of eliminate death from preventable diseases.
Private sector example
Using the private sector let us take the example of a mobile application that helps students organise their college’s lives. It generates revenue and has a product roadmap with several exciting features planned. The organisation decides it is will adopt an EDM framework and defines its strategic goal as:
To ensure all students regardless of background or disability to get the support they need to complete their education.
Let us consider what is the impact on the product road map by adopting this specific strategic goal.
A new proposed feature within the roadmap matches with the intermediate goal and the long strategic goal of the organisation.
- There is an existing tactical goal can be fulfilled or partial fulfilled by building this feature. The product manager must decide the priority of this feature relative to other items in the backlog.
- There is no existing tactical goal to match the new proposed feature. The product manager must decide if they should recommend that a new tactical goal be created now or is better to complete the current tactical goals before looking at the possibility of creating new ones.
If the new proposed feature does match any intermediate goals of the organisation it should remain in the product backlog until it can.
The other aspect to consider as it builds it looks to get feedback often and early to ensure what is developed is inline with the user needs.
Measurement within EMB
In the previous section we showed how Evidence-Based Management can be used to determine the goals of the organisation and how through an agile framework of testing we can work towards to fulfilling those goals.
In this section we are going to look at how EMB recommends measuring the value of the organisation and how the organisation can therefore optimise for value.
Organisations may measure value by looking at:
- Activities: these are things that individuals do. For example, an individual may design an image, write code, attend meetings or perform tasks.
- Outputs: these are things that the organisation produces. It could be reports, products, new product features or even scientific discoveries.
- Outcomes: Outcomes are of two types: positive and negative. Positive outcomes happen when our customers or users benefit from using our product or service. For example, establishing a platform for C2C (consumer to consumer) trade like eBay or Etsy allows consumers to trade. A user of these platform benefit being able to generate an income stream. By focusing on the user needs, the positive outcome for the user is also a positive outcome for the company. Negative outcomes are undesirable and can happen when a user is no longer able to achieve something they were previous able to. Sometimes an organisation may retire a product or service which can cause a negative outcome.
EBM is focused on outcomes rather than outputs or activities. It believes that true value is realized by serving the user and not by producing more product or doing more activities. By focusing on the user, we build a product that our users want more then they just need. That in turn improves customer retention and customer referrals.
How is value measured within Evidence-Based Management?
Value can be measured in multiple different ways, some looks at EBITA, others look at ROI whilst others may care about NPS. EBM defines value in four different areas, these are called Key-Value Areas (KVAs). They are:
- Current Value (CV)
- Unrealized Value (UV)
- Ability to Innovate (A2I)
- Time to Market (T2M)
Current Value (CV)
The current value is a measure of what the organisation can deliver to its customers and stakeholders today. Current value is not limited to revenue generated or size of the paid database but also considers the levels of happiness of the customers, the staff and the investors/stakeholders.
Questions they can be used to assess it values could include:
- How happy are our employees? Would they recommend working there?
- How happy are our users/customers with our product or service? Is their happiness on the incline or decline?
- How happy are our investors and stakeholders? Is their happiness on the incline or decline?
By measuring employee happiness, we can optimise for it because an incline or decline in the level of happiness has ramifications all across the organisation. An engaged and happy employee is going to contribute more to the company success then an employee who is unhappy and disengaged with the company.
Levels of user happiness for the product or service acts as an indicator of what needs to be improved in the product to retain customers. Looking at how often a feature is used, and for how long, is a way to understand what a customer value in the product. It is also important to compare user level of happiness to the past to determine if it is on the incline or decline.
ROI is important for investors but so is the direction and the vision of the organisation. If they are concerns, then they must be addressed. Continuously engaging stakeholders to determine what concerns them and then working to deal with the concerns is essential for the success of an organisation. By measuring for it, we can optimise for it.
It is important that an organisation continuously reassess its position.
Unrealized Value (UV)
Current value measures the value of the organisation today, unrealized value is the potential future value of the organisation. These values represent the difference between what is possible today and what is possible in the future. This difference is the unrealized value.
A start-up may have high unrealized value and a low realized value as it is recently started its journey to greatness. Investing at this stage can help unlock the Unrealized Value and increase the Current Value of the organisation.
An organisation within a mature market that is either a monopoly or an oligopoly may have a high realized value but a low unrealized value as the opportunity to grow is limited.
The type of questions that an organisation can use to assess and reassess its unrealized value could include:
- Is it possible for the organisation to generate additional value in this market or another market?
- What is the opportunity cost of pursuing these opportunities?
- What level of investment would be required to realize the unrealized value?
Time-to-Market (T2M)
An organisation may build an amazing product or service but not until it is available on the market can it deliver value to the customer. Time to market is a measure of how quick an organisation can deliver new products, new services or develop new capabilities. What is important for an organisation is to improve or sustain its time to market.
By maintaining a schedule of regular releases with maybe a smaller number of features per release can improve T2M as well as the Current Value of the organisation relative to having larger releases with more features but release less often.
Having the ability to deliver a product on time but then spending several months fixing the problems that the release created limits the ability for the organisation to launch new products.
Questions that the organisation can ask to assess their T2M can include:
- How fast can an organisation test a hypothesis with users?
- How fast can the learnings from the test be understood and adopted by the organisation?
- How fast can the learnings from the test be implemented?
It is important to optimise the time to market to ensure what is learnt can be implemented in the time frame where it can add value. Being late to market may mean a new feature is no longer consider as adding value but rather becomes what is expected but still important, to no longer required as users has moved to another solution that is more preferable.
Ability to Innovate (A2I)
Innovation is the ability to improve a product or service for its users. The ability to innovate is a measure of ease at which an organisation can innovate.
The type of questions that an organisation can use to assess and reassess its unrealized value include:
- What are obstacles that stop an organisation from innovating?
- What can be done to remove these obstacles?
- What are the factors which can impacts users from benefiting these innovations?
- What can be done to ensure users benefit from these innovations?
It is important for organisation that follows a hypothesis, experiment & measure, inspect and adapt model to improve the organisation value. Competitors may be a good example of what is possible, but the real competition is about constantly growing by asking what could you do better.
The interaction between the four values
Realized value is the unrealized value in the past that was achieved and acts as an indicator of how the organisation can innovate and the blockages and stumbling blocks it circumnavigated during its journey.
Unrealized value carries risk as there is no guarantee of unlocking it. By breaking down the unrealized value into tactical and immediate goals as we achieve each goal, we can unlock unrealized value. By adopting a short iterative approach, we can ensure if something does not yield the results we are looking for or is no longer valid, we simply try something else or pivot in a totally different direction. This helps us to achieve our goals and saves time and resources by not pursuing activities that do not yield the optimal results.
Our ability to innovate is how we can unlock unrealized value. Innovation is seen by some by a single eureka moment. Innovation under an Agile is a process by where we test multiple different hypothesis and look to what works best, and once that is found something we continue to optimise it further through further iterations. The speed of innovation is depending upon our time to market. Optimising time to market, increase the number hypothesis we can test, which helps realize the unrealized value that is available.
To learn more EBM I would recommend reading the guide, which can be found here.
