May 7th 2020 marked the first Inside Xbox event to show off gameplay running on the Xbox One X Series, Microsoft’s latest gaming console. There was much hype surrounding this event, unfortunately, many felt disappointed by what was shown. Aaron Greenberg the General Manager of Games Marketing had to say about this:
Had we not said anything & just shown May Inside Xbox show like we did last month, I suspect reactions might have been different. Clearly we set some wrong expectations & that’s on us. We appreciate all the feedback & can assure you we will take it all in & learn as a team. 🙏🏻💚
— Aaron Greenberg 🙅🏼♂️❎ (@aarongreenberg) May 8, 2020
Is failure bad?
We live in a world where success is celebrated and failure is seen by some as a disaster. Not scoring that point, not passing that exam, or getting that deal can hurt but it can also stop someone from trying again.
“I have not failed. I’ve just found 1,000 ways that won’t work.” Thomas Edison.
Thomas Edison is regarded as America’s most recognised and prolific inventor. In his pursuit in developing an efficient incandescent lamp, he tried over 1,000 different ways with each one failing, only to finally succeed. Sometimes in life, the greatest opportunity to learn comes from failing, getting up and trying again.
In Agile, failure is just a result of a test, there is no social stigma or negativity around it. By failing and you have learnt something you may not have learnt otherwise.
“What good is an idea if it remains an idea? Try. Experiment. Iterate. Try again. Change the world.” Simon Sinek Together is Better
Failure in an Agile Environment
The purpose of this post is to show how we can leverage user research coupled with a testing mindset to help to exceed user expectation and create a buzz around the Xbox One Series X and its games.
Agile can be defined in two different ways:
- The ability to react to an ever-changing environment. When technology introduces new opportunities, markets collapse, or new markets open up, having the ability to react quickly amd take advantage of the opportunity is being agile.
- To take what you don’t know (the known unknown) and use a systematic approach to test different hypothesis to make it become a known known.
In this post we will look at the second approach, that is to develop a set of hypotheses, build a minimum viable product (MVP) to test and use the results for either iterative or deliver to the end users.
The importance of user research
It is sometimes very easy to forget the maxim,
“you are not your customer”
To help eliminate such biases, user research helps to understand user behaviour, their needs, their motivations through testing, observation and interviewing.
What is user research within Agile?
If Agile is the process by which we can develop and test new ideas, user research is the way we can help generate ideas to test from the users.
What are we looking to test?
Jerret West, CVP of Gaming Marketing at Xbox said, “starting with the May 7th episode of Inside Xbox, we will be showcasing what happens next in the world of Xbox, every month, which is why we’re calling it ‘Xbox 20/20.’”
By having an event every month until release provides the perfect opportunity to test and iterate.
To illustrate the idea of using user feedback and iterating we will look at two different examples:
- Ray Tracing – a hardware feature for the Xbox Series X.
- Halo Infinite – the launch of a new game.
Hardware – Ray Tracing on Xbox Series X
The Xbox Series X is the first Microsoft console to feature Ray Tracing which has traditionally been reserved for high-end PCs.
What questions do users have and where does the information exist?
Gathering information online
In this example, we are looking for users who are discussing ray tracing. This may include content on Quora.com, Twitter, Microsoft Answers and the comment section in YouTube videos. What is important is to identify what questions are people asking? What makes people excited about Ray Tracing? And how are people justifying the purchase of a new graphic card or a next-generation console to experience Ray Tracing?
Asking questions – slicing and dicing data
We have all seen Twitter polls, voting in Instagram Stories, emails with questionnaire etc. there are many ways to get answers to questions. We may want to ask questions to specific audiences which can be done by using different social accounts, time of day of posting or paid postings where we have even more control on who participates.
What if we took this a stage forward? Microsoft has significant amount of user data on which games are played, which apps are used and which consoles have been purchased by user. We could create different audience segments:
- Users who have purchased Xbox S and Xbox X.
- Users who buy multiple games per month.
- Users who play less than 4 hours of gamer per month.
- Users by gamerscore.
It can then be possible to target user in console (having surveys on the dashboard) or over email. Questions can be asked can include if they know what Ray Tracing is? How important are graphics relative to other feathers a new console has to offer? Or if they have any specific questions on Ray Tracing?
In terms of the audience we can take two approaches:
- Select the audience which is most likely to purchase the Xbox Series X and target those.
- Look for the audience that share similar properties which is largest in in size relative to other audiences. By optimising a campaign for this audience will also have the greatest impact on the whole audience.
Developing a hypothesis and MVP
Now that you have data, it is about developing hypothesis from the data and finding, ‘quick and dirty’ ways to test. What is important to remember:
- A hypothesis is just an idea, not every idea will work. Don’t become attached to an idea until you have data to prove its validity.
- Your hypothesis may not be the best.
- It is all about getting feedback and iterating.
- The solution may not fit all (e.g. ways people learn or consume media.)
- The endpoint is a decision based upon on data you currently have on how to present Ray Tracing for the Series X launch period. Things change over time.
Let us for a moment walk through an example to see how this could work. We may for example have a segment of the audience who does not know what Ray Tracing is but regard improved graphics as the most important factor when buying a new generation console.
We could test showing videos of Ray Tracing on and off versus screenshots of games with and with Ray Tracing, to see which one is regarded as a better way to communicate how Ray Tracing can make graphics look better? Taking that a stage by iterating what if we are testing showing videos/screenshots of games that audience commonly play with Ray Tracing on and off vs games videos/screenshots of new not played games with Ray Tracing on and off i.e. do user identify Ray Tracing better with games they recognise or newer games.
If Ray Tracing from games people commonly play resonate more, then the reveal event of the Xbox Series X should take that into considering when delivering the show,
Launching a New Game – Halo Infinite
Halo is a system seller and the second biggest IP after Minecraft. We know Halo Infinite will be shown at the Xbox 20/20 event in July and the game is due out this year.
Let’s set ourselves a different question, post the reveal event how quickly can we adapt to user feedback and produce marketing material? Usually, in any marketing campaign, we define all the marketing materials required in advance, have them ready for the event in advance. What if we took a different stance, and decided we needed to be flexible and be able to rapidly respond to user feedback during the reveal.
As an analogy, a good chef has a well stock larder to accommodate potential changes in a menu, so what if we built up a well-stocked larder in advance for the Halo Infinite reveal?
We are going look at two sources to try to predict how fans will react to the reveal. Internal users and historical reactions to other games.
Have employees to Xbox/Microsoft who are experienced in storytelling but have not worked on the Halo reveal in a ‘room’ together to show the Halo reveal to. The objective of showing the reveal is to highlight potential problems with the reveal.
I would recommend using the format called ‘Braintrust’ pioneered by Pixar, where everyone’s’ opinion is equal, no one is to be blamed for failure and people are encouraged using Radical Candor to identify the root causes of problems. To learn more about Braintrust I would suggest reading this article on fastcompany.com or reading the book Culture Code by Daniel Coyle
Having this critical feedback can be used to identify which problems need to be resolved before the reveal and which are a potential problem that we need to help mitigate during the reveal event. As a simple example to illustrate it could be, it would be great to see to learn more about weapons Master Chief has at his disposal.
There have been many FPS in the past, there include previous releases of Halo including the more recent release of Halo Master Chief Collection on the PC. There have had reveals, trailers, gameplay videos with lots of user feedback. Take all that user feedback and identify common questions, common disappointments, recommendation I.e. ‘I wish we could see more of.’
Building a multi-disciplined team
The idea is to put together a team with different skills so they can act independently from other teams, this may include designers, content writers, engineers, video editors etc.
Their first objective is to understand what the Halo Infinite reveal contains, take the feedback from the internal users and the data from the historical reactions and understand possible pitfalls.
Building a content library
Once you have all these possible scenarios, it is then about ranking them according to the chance of it happening on the day of the reveal and the effort required to produce something. Let’s look at a possible example of this playing out.
The July event is 90 minutes long. Halo Infinite is shown in the first 5 minutes and there is space reserved for the last 5 minutes for a follow up to the Halo Infinite reveal. Within minutes of the reveal feedback coming online indicates something as an area of disappointment, for example, the lack of multiplayer gameplay. You have about 80 minutes event to rectify it and present something at the end of the show.
It may seem scary at first producing a video in a very limited time-space but with all things, it is all about practice and having retrospective to see how you can improve. Questions that could come up in the retrospective could include:
- What additional content needs to be added to the content library?
- Which Halo staff need to available to interview on the day if needed?
- What skills does the team need to learn?
- How does the team perform under pressure?
- What are the impediments that need to be removed?
- What backs can we put in place?
Each time you run a scenario the team become quicker because they know what works, and they have through the retrospective dealt with the bottlenecks.
This post looks using an empirical iterative test approach pre and post an event to see how we may react to user feedback to help produce the best possible marketing event.
Thanks to Zohre Nemati for sharing their work on Unsplash.